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ABSTRACT 


Improvised Explosive Devices (lEDs) will continue to evolve as the enemy’s weapon of 
choice. Dynamic and responsive enemies alter lEDs in reaction to our Counter-IED 
efforts, which results in the need for continuous accelerated testing and acquisition of 
Counter-IED tools. Over the last five years, an accelerated acquisition process enabled 
the expansion of testing entities and ranges beyond the purview of the Joint Improvised 
Explosive Device Defeat Test Board (JTB) established to monitor them. As a result, 
information sharing between JTB organizations is not efficient, which leads to less 
capable and slowly fielded equipment. This research reports on the use of a cognitive 
task analysis and contextual inquiry focused on identifying and mitigating information 
sharing impediments. The objective is to provide recommendations to improve 
information sharing between personnel involved with the JTB. Results indicate that the 
most common information sharing barriers relate to organizational culture, which leads to 
the recommendation of creating a JTB position specifically focused on information flow 
inside and outside of the organization. Results regarding information sharing enablers 
indicated that enhancing test range cooperation and improving information systems 
through more tester level meetings, use of the customized affinity diagram, and 
implementing user recommendations would improve information sharing. 


V 



THIS PAGE INTENTIONALLY LEET BLANK 


VI 



TABLE OF CONTENTS 


1. INTRODUCTION.1 

A, ESTABLISHING A FOCUS.1 

1. Background.1 

2, Research Questions.2 

IT BACKGROUND.3 

A. CONTEXT.3 

B. INFORMATION SHARING: WHAT IT IS AND WHY PEOPLE DO 

IT.3 

1. What It Is.3 

2, Why People Do It.4 

C. ORGANIZATIONAL CONSIDERATIONS: HOW WE 

ENCOURAGE IT.5 

1, Organizational Culture.5 

2, Organizational Environment.7 

3, Organizational Structure.10 

D. COGNITIVE SYSTEMS ENGINEERING: HOW TO FACILITATE 

IT.12 

1. Introduction to Cognitive Systems Engineering.12 

2. Critical Decision Method.12 

3. Contextual Inquiry.13 

E. SUMMARY.14 

1. Context.14 

III. METHODOLOGY.17 

A. RESEARCH METHODOLOGY.17 

1. Commencement.17 

B. PREPARATION.18 

1. Concurrent Processes.18 

2. Participant Selection.18 

3. Rehearsals.18 

C. INITIAL DATA COLLECTION.19 

1. Interviews and Surveys.19 

a. Overview . 19 

b. Initial Data Collection Summary . 22 

D. INITIAL DATA ANALYSIS.22 

1. Overview.22 

2. Consolidation.23 

3. Grouping.23 

E. FOLLOW-UP METHODOLOGY.24 

1. Method Determination.24 

F. FOLLOW-UP PREPARATION.24 

1. Participant Selection.24 

vii 









































G. FOLLOW-UP DATA COLLECTION.25 

1. Follow-Up Interviews.25 

a. Follow-Up Interview Summary . 26 

2, Follow-Up Surveys.27 

H. FOLLOW-UP ANALYSIS.27 

1. Consolidation and Grouping.27 

I. CONSOLIDATED ANALYSIS.27 

1. Affinity Diagram.27 

a. Establish a Focus . 28 

b. Record Key Aspects . 28 

c. Tier III . 29 

d. Tier II. . 29 

e. Tier 1 . 30 

f. Record the Diagram . 30 

2. Surveys.30 

a. Qualitative Data . 30 

b. Quantitative Data . 30 

J. SUMMARY.31 

1, Overview.31 

2, Data Collection.31 

a. Interviews . 31 

b. Surveys . 31 

3, Data Analysis.32 

a. Qualitative . 32 

b. Quantitative . 32 

IV. RESULTS.33 

A. OVERVIEW OF RESULTS.33 

1. Presentation of Results.33 

2. Consolidated Demographics.33 

3. Interview Summary.34 

a. Initial Interview Summary . 34 

b. Follow-Up Interview Summary . 37 

B, QUALITATIVE RESULTS.41 

1. Cognitive Task Analysis.41 

a. CTA Table Description . 41 

2. Affinity Diagram.48 

a. Background . 48 

b. Affinity Diagram Results . 49 

C. QUANTITATIVE RESULTS.55 

1. Consolidated Information Sharing Survey Results.55 

D, CONCLUSION.57 

1. Return to the Purpose.57 

V. CONCLUSIONS.59 

A, OVERVIEW.59 

B, CTA CONCLUSIONS.59 

viii 
















































1. Barriers to Information Sharing.59 

2. Enablers for Information Sharing.60 

3. Participant Recommendations for Improvement.61 

C. AFFINITY DIAGRAM CONCLUSIONS.61 

1. Levels of Abstraction.61 

2. Affinity Diagram Conclusions.62 

D. INFORMATION SHARING SURVEY CONCLUSIONS.62 

1. Organizational Information Sharing.62 

2. Intra-Organization Communication Structure.63 

3. Organizational Culture.63 

E. RECOMMENDATIONS.63 

1, Overall.63 

2, CTA Recommendations.63 

3, Affinity Diagram.64 

4, Information Sharing Survey.65 

5, Future Research.65 

LIST OF REFERENCES.67 

APPENDIX, INFORMATION SHARING SURVEY TEXT.69 

INITIAL DISTRIBUTION LIST.75 






















THIS PAGE INTENTIONALLY LEET BLANK 


X 



LIST OF FIGURES 


Figure 1. Example of a Hierarchical Organization.11 

Figure 2. Example of a Nonhierarchical Organization.12 

Figure 3. Example of an Affinity Diagram.29 

Figure 4. Shell of the Affinity Diagram.48 







THIS PAGE INTENTIONALLY LEET BLANK 



LIST OF TABLES 


Table 1. Initial Interviews by Organization.22 

Table 2. Initial Interview Information Sharing Section Summary.34 

Table 3. Initial Interview Policy and Procedure Section Summary.36 

Table 4. Follow-Up Interview Summary: Specific Information That Needs to be 

Shared.37 

Table 5. Follow-Up Interview Summary: Information Visibility Between 

Organizations.38 

Table 6. Follow-Up Interview Summary: Organizational Structure.39 

Table 7. Follow-Up Interview Summary: Specific Information to be Put Into a 

Central Repository.40 

Table 8. Follow-Up Interview Summary: Security Classification Effects on 

Information Sharing.40 

Table 9. Motivational Barriers.41 

Table 10. Organizational Culture Barriers.42 

Table 11. Organizational Environment Barriers.43 

Table 12. Organizational Structure Barriers.44 

Table 13. Motivational Enablers.44 

Table 14. Organizational Culture Enablers.45 

Table 15. Organizational Environment Enablers.45 

Table 16. Organizational Culture Recommendations.46 

Table 17. Organizational Environment Recommendations.47 

Table 18. Tier III Diagram: Personal Relationships and Individual Perceptions.49 

Table 19. Tier III Diagram: JTB Eevel Issues and Eeadership.50 

Table 20. Tier III Diagram: Inter-Organizational Communication and Best Practices ...51 

Table 21. Tier III Diagram: Standards and Points of Contact.51 

Table 22. Tier III Diagram: Personnel Turnover and Quality of Shared Information.52 

Table 23. Tier III Diagram: Information Sharing Tools.53 

Table 24. Tier III Diagram: Access to Information and Information Visibility.53 

Table 25. Tier II Diagram: Individual Issues, Eeadership Considerations, 

Management, Information Sharing, and Information Availability.54 

Table 26. Tier I Diagram: Individuals, Eeadership, and Information.55 

Table 27. Consolidated Information Sharing Survey Results.56 


xiii 





























THIS PAGE INTENTIONALLY LEET BLANK 


XIV 



LIST OF ACRONYMS AND ABBREVIATIONS 


APG 

Aberdeen Proving Grounds 

C-IED 

Counter Improvised Explosive Device 

China Lake 

Naval Air Warfare Center Weapons Division China Lake 

CTA 

Cognitive Task Analysis 

DoD 

Department of Defense 

DoD Strategy 

Department of Defense Information Sharing Strategy 

lED 

Improvised Explosive Device 

JIEDDO 

Joint Improvised Explosive Device Defeat Organization 

JTB 

Joint Improvised Explosive Device Defeat Test Board 

National Strategy 

National Strategy for Information Sharing 

NFS 

Naval Postgraduate School 

PM 

Program Manager 

Point Mugu 

Naval Air Warfare Center Weapons Division Point Mugu 

QDR 

Quadrennial Defense Review 

TSWT 

Theater Support Web Tool 

YPG 

Yuma Proving Grounds 


XV 



THIS PAGE INTENTIONALLY LEET BLANK 


XVI 



ACKNOWLEDGEMENTS 


I would like to thank Katie and Ava for their unwavering support and endurance 
of lost evenings and weekends. 

I would also like to thank my advisors, Professor Susan Hutchins and Scot Miller, 
CAPT, USN (Ret.), for their initial direction and continued guidance throughout the 
process. 


xvii 



THIS PAGE INTENTIONALLY LEET BLANK 



I. INTRODUCTION 


A, ESTABLISHING A FOCUS 

1, Background 

On February 14, 2006, the Joint Improvised Explosive Device Defeat 
Organization (JIEDDO) was established. Subcomponents of JIEDDO include multiple 
boards, one of which is the Joint Improvised Explosive Device Defeat Test Board (JTB). 
According to Department of Defense Directive 2000.19E, “The JTB coordinates and 
synchronizes all Joint lED Defeat test and evaluation (T&E) events within the 
Department of Defense and assists the Military Departments to maximize utility and 
decrease redundancy in testing of Joint lED Defeat initiatives” (p. 16). Due to the 
complexity of the environment, structure of the organization, socio-cultural factors, and 
lack of incentive to share knowledge among test conductors, the JTB is currently not 
accomplishing its mission as efficiently as possible. 

Improvised explosive devices (lEDs) continue to be the enemy’s weapon of 
choice. The dynamic and responsive enemy alters lEDs in reaction to our Counter lED 
(C-IED) efforts, which results in the need for continuous accelerated testing and 
acquisition of C-IED tools. 

Over the last five years of this accelerated acquisition process, testing entities and 
ranges have expanded beyond the purview of the JTB. Personnel performing functions in 
support of the JTB have little incentive to share knowledge. In fact, due to competition 
for funding sources, in some cases, there is a disincentive to share information among test 
ranges. 

In an effort to improve information sharing within the JTB, the Naval 
Postgraduate School (NPS) has embarked on a research campaign to conduct thesis 
research in order to help make the JIEDDO test program more successful. A knowledge 
engineering effort was conducted with the goal of developing an understanding of the 
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critical tasks and required information types involved in the C-IED process. The final 
objective is to improve information sharing between all personnel involved with the JTB. 
This knowledge engineering effort and a related end-user analysis are the first two in the 
series of six theses planned as part of this campaign. They may be followed by theses on 
JTB portal design, a second cognitive task analysis on three-dimensional user interfaces, 
a C-IED optimization tool, and a three-dimensional JTB portal design. 

It is important to understand that there is a difference between the composition 
JTB, as defined in DoD Directive 2000.19E, and the enterprise of organizations and 
individuals that process and conduct research and testing to accomplish the stated 
mission. The scope of this thesis research was limited to information sharing regarding 
test execution and reporting for the JTB. Interviews were conducted with: (1) individuals 
within the JTB; (2) members of the advanced communications and foreign release 
working groups; (3) test directors at Yuma Proving Grounds (YPG), Aberdeen Proving 
Grounds (APG), and Naval Air Warfare Center Weapons Division China Eake (China 
Eake); and (4) other subject matter experts with JTB experience. Interview questions 
focused on what information each individual needed, expected, received, failed to 
receive, and provided. 

This research was required because the lack of information sharing between these 
entities contributed to duplication of effort, inefficient use of resources, and most 
importantly, delays in fielding equipment to the war fighter (U.S. Government 
Accountability Office, 2009). Typically, the JTB would send requirements, 
prioritization, coordination, and resources, and receive analysis, and reporting from 
working groups, program managers (PMs), and Joint Program Executive Offices. As of 
the writing of this thesis, information did not flow as needed for all personnel associated 
with testing C-IED weapon systems. 

2, Research Questions 

In the conduct of this thesis, the researcher focused on two questions. What 
impediments exist to information sharing within the JTB? How can those impediments 
be mitigated? Multiple secondary supporting questions were also considered. 
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II. BACKGROUND 


A, CONTEXT 

The objective of this thesis was to improve information sharing in the JTB 
organization; the method to achieve that objective was a cognitive engineering analysis. 
It describes how users decide what information to share and with whom. Before 
considering their decision-making process, however, one must have an understanding of 
the concepts that will be discussed in the following chapters. This chapter provides a 
review of information sharing, organizational considerations, and cognitive systems 
engineering. 

B, INFORMATION SHARING: WHAT IT IS AND WHY PEOPLE DO IT 

1. What It Is 

Although information sharing seems like a straightforward concept, it has 
different connotations that depend on the context of the discussion. For example, in 
politics it often refers to interagency coordination, while in the medical field it refers to 
privacy of individual medical records. Because this project is focused within the 
Department of Defense (DoD), the definition and understanding of information sharing 
will be based on the Department of Defense Information Sharing Strategy (DoD 
Strategy). The DoD strategy is based on the National Strategy for Information Sharing 
(National Strategy) and the 2006 Quadrennial Defense Review (QDR); it is implemented 
through the DoD Information Sharing Implementation Plan. The National Strategy is 
focused on combating terrorism. The vision is stated, in part, to “establish a more inte¬ 
grated information sharing capability to ensure that those who need information to protect 
our Nation from terrorism will receive it and those who have that information will share 
it” (National Security Council [NSC], 2007, p. 2). It does an excellent job of providing 
guidance to that end; however, the closest that it comes to defining the concept is, “The 
exchange of information should be the rule, not the exception, in our efforts to combat 
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the terrorist threat.” (NSC, 2007, p. 1) What makes it relevant to this thesis is that it 
influenced the creation of the DoD Strategy, which broadens the vision and provides a 
working definition of the concept. The DoD strategy was published in response to the 
QDR decision to, “Strengthen its data strategy ... to improve information sharing and 
information assurance, and extend it across a multitude of domains . . . [and] Develop an 
information-sharing strategy to guide operations with Federal, state, local and coalition 
partners.” (U.S. Department of Defense [DoD], 2006, p. 59). 

The DoD Strategy broadened the vision from terrorism to one that is applicable 
throughout its area of influence; the stated vision is, “ Deliver the power of information 
to ensure mission success th rough an agile enterprise with freedo m of 
maneuverahility across the information environment” (U.S. Department of Defense 
[DoD], 2007, p. 3). (Bold text and underlining are included in the original.) The DoD 
Strategy definition of information sharing, which will be the standard throughout this 
thesis, is “‘Making information available to participants (people, processes, or 
systems).’ Information sharing includes the cultural, managerial, and technical behaviors 
by which one participant leverages information held or created by another participant” 
(DoD, 2007, p. ii). (The quotation marks and bold text are included in the original 
document.) It is significant to note that the definition specifies information sharing as a 
behavior, which means that it cannot be relegated to a task of an information system. The 
Strategy includes appropriate changes to the culture and approaches to implement, but it 
does not address what motivates people to share. 

2. Why People Do It 

The culture will not change, nor will the approaches be successful if individuals 
are not motivated to share and if they do decide to share, there is a difference between 
sharing information and sharing tacit knowledge. Research by Lam and Lambermont- 
Ford (2010) indicates there are three categories of motivation to share information: 
intrinsic normative, intrinsic hedonic, and extrinsic. Intrinsic normative motivation is 
sharing because one feels that it is the right thing to do. In the case of the research topic 
for this thesis, a member of the JTB organization who is motivated by intrinsic normative 
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motivation would share information because the person would consider it a job 
requirement. Intrinsic hedonic motivation involves sharing information because it makes 
one feel good. In this example, the member feeling intrinsic hedonic motivation would 
share information because it would make the person feel good that war fighters may 
benefit from it. Extrinsic motivation is sharing information in return for something of 
value. The member in this case would share information if it would result in a 
promotion. What makes motivating an enterprise to improve information sharing is that 
different members are motivated by different permutations of the categories and the 
permutation changes for each member changes throughout the day. 

To further complicate the situation, the DoD definition specifies sharing of 
information held or created by another participant. Information held by another can be 
considered tacit knowledge or an information product. Tacit knowledge, similar to 
expertise, is understood without being part of a document. Information products, on the 
other hand, are tangible documents. People view the sharing of these two types of 
information differently. Essentially, the permutations of the motivational categories are 
different, so one may share an information product because of intrinsic normative 
motivation, but require intrinsic hedonic motivation to share tacit knowledge. Another 
difference between these types of information is the medium used for sharing. 
Information products are easily shared via information systems. The unwritten nature of 
tacit knowledge, on the other hand, makes it difficult to document and disseminate. 
According to research by Constant, Kiesler, and Sproul (1994) and Holste and Eields 
(2010), tacit knowledge sharing also varies with the level of trust the individual has with 
his coworkers more than sharing information products. The higher the level of trust, the 
more tacit knowledge is shared. 

C. ORGANIZATIONAL CONSIDERATIONS: HOW WE ENCOURAGE IT 

1. Organizational Culture 

Considering the complexity of different motivations for sharing information, an 
organization must create a diverse culture in order to stimulate as many members as 
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possible to participate. The individual stimulation must be generated through an 
organizational culture that is strong enough and focused enough to influence a member’s 
mindset. Although it is true that “successful information sharing necessitates a mindset 
where information is continually shared as a normal course of work,” (DoD, 2007, p. 5) 
establishing that mindset requires a strong supportive organizational culture. There are 
many definitions of organizational culture, and most are founded on an organization’s 
values, beliefs, and norms. For the purpose of this thesis, the value considered is the 
sharing of information. Beliefs and norms will be addressed through policy and process, 
respectively. 

Beliefs for any organization must be established by an authority figure and 
disseminated to the members. Because it originates from an authority figure, members 
accept it as legitimate and a single authoritative source provides the common description 
of the belief. To put it plainly, “to encourage sharing of work information, organizations 
need a clear policy.” (Constant et ah, 1994, p. 418) As the initiation of the 
implementation of the value to share information, a policy must not only exist, it must 
provide “clear, concise, and comprehensive guidance” (DoD, 2007, p. 11). Depending on 
the organizational structure, multiple policies may be required at different levels or 
knowledge hubs. It is essential that all policies are synchronized to avoid conflicting 
guidance (DoD, 2007). 

The policy must direct members to take specific standardized actions, or norms, 
though the processes that it directs. While the most important aspect of information 
sharing is motivating the individuals to participate, the organizational leadership must 
provide a standard process or set of processes to enable the members to take action and 
streamline evaluation. As a result, they will be able to take uniform action toward a 
common goal. In order to be effective, the processes must be created with the user in 
mind, assigned, enforced, and reviewed for barriers to their purpose. 

When creating the new processes, relating them to existing procedures or tasks 
will ease the transition to the new process. It will also speed the cultural assimilation or 
“user buy-in” that is necessary to be effective (McLaughlin, 2010). 
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Just as the policy must be clear and easy to understand, the processes it directs 
must be assigned to specific member positions for responsibility of compliance and 
communication (McLaughlin, 2010). Ideally, because the process is related to current 
practice, compliance should not be much of an issue. However, the assigned individual 
must have the authority and ability to enforce compliance when necessary. The 
responsible member must also serve as the point of contact for clarification and 
suggestions for improvement. Even in the ideal situation presented previously, 
enforcement is required. 

Whether members lack the motivation to share, make errors, or deliberately 
intend to neglect the policy, the responsible member will need to enforce compliance 
with the process. The manner of enforcement will depend on the nature of the policy and 
procedure, lack of compliance, organizational structure, and multiple other variables to be 
deliberated by the responsible member. 

Following implementation, the processes must also be reviewed at regular 
intervals in order to ensure that they are still applicable and identify barriers to 
compliance (McLaughlin, 2010). As organizations evolve, their processes must change to 
maintain applicability to the spirit and/or letter of the policy and evoke the organizational 
culture. Considering the nature of interaction required for information sharing, if one 
element of the organization encounters a barrier and fails to participate, the entire 
organization is affected. By identifying barriers to compliance, they can be addressed 
and mitigated to the greatest possible extent. Identifying the barriers to sharing and 
recommending ways to mitigate them is the central focus of this thesis. 

2, Organizational Environment 

Once the organizational culture has been established and conditions for 
motivating members to share information have been developed, the tools for members to 
conduct the act of sharing must be implemented. Social interaction, incentives, and 
information systems are the tools that create the organizational environment. 

The social interaction aspect of the organizational environment is the most direct 

means to motivate information sharing. Methods of implementation include exploiting 
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social networks, increasing interaction, and increasing eollaboration. Exploiting soeial 
networks essentially puts the power of peer pressure to work. By studying the soeial 
networks of an organization, it is possible to identify members who aet as sharing hubs 
within their network, bridges to other social networks, and links to providers (Sun, 2010). 
Exploiting members in those unoffieial positions will create positive effeets beeause 
people will be comfortable in their roles and fewer new relationships will have to be 
established than if social networks were ignored. 

Something as simple as including greater soeial interaction can improve the 
environment. Ironically, as individual communication technology continues to evolve, 
people spend less time communicating face to face. In light of that trend, researeh has 
found that “the best organizational policy may be simply to create oecasions for people to 
talk and exchange knowledge, opinions, and advice” (Constant et ah, 1994, p. 419). The 
organizational environment may be improved, then, by having more meetings, training 
members on how to conduct meetings more efficiently, and/or increasing participation in 
large professional gatherings like conventions and symposiums. Sharing tacit knowledge 
may also increase because McAllister’s research (Holste & Eields, 2010) has shown that 
“higher levels of affiliative behavior (taking a personal interest, passing on information, 
helping another with tasks) inereased affect-based trust among professional peers” 
(Holste & Eields, 2010, p. 136). 

Increasing collaboration among members improves the organizational 
environment by increasing trust and incorporating social interaction. The ability of an 
organization to eollaborate on a large scale requires a well-established operational 
environment, but benefits can also be obtained through small scale collaboration 
exercises. The benefits are greatly based on trust, which, according to McAllister and 
Dietz (Holste & Eields, 2010), “may be inereased through frequent direct engagement of 
co-workers in eollaborative proeesses—especially situations that illustrate 
interdependency and provide opportunity for workers to demonstrate individual 
competency (Holste & Eields, 2010, p. 136). The aspeet of direct engagement requires 


8 



social interaction. The benefits of physical social interaction have been presented, but the 
research did not put a physical restraint on direct engagement. Information systems could 
be incorporated into the collaborative process as well. 

Many organizations decide to include incentives in the environment to 
extrinsically motivate members to share information. In order to be effective, the 
incentives must fit the type of information to be shared and the tools available (Hansen, 
Nohria, & Tierney, 1999; Sun, 2010). For example, if the goal of the organization is for 
the members to share more information products, the incentive should be related to the 
number of products created or entered into an information system. If the goal is to 
increase the sharing of tacit knowledge, incentivizing document creation would be 
ineffective. Rather, the incentive should focus on productive social interaction which 
results in the passing on of tacit knowledge. 

Information systems are often seen as the silver bullet for solving information 
sharing issues. Although they are a significant aspect of the operational environment and 
greatly enhance an organization’s ability to share information, they are only as good as 
the members who operate them. When considering including an information system in 
an operational environment, the planners must consider whether to use a codification or 
personalization strategy. When studied from the proper perspective, as tools that 
members use to share information, when appropriately incorporated into the strategy, 
their contribution falls into one of two categories: social interaction or data manipulation. 

While all information sharing strategies are unique, they can be categorized as a 
codification or personalization strategy. In codification, information is “extracted from 
the person who developed it, made independent of that person, and reused for various 
purposes” (Hansen et ah, 1999, p. 108). An information system with a large repository to 
mine for information would be appropriate for this type of strategy. In personalization, 
on the other hand, the information is largely tacit, so a large repository would not be 
appropriate. An information system for that strategy would be more effective if it were 
based on interpersonal communication and included tools for collaboration. Of course all 
organizations will have some aspects of each category in their strategy, but only one 

should be followed to ensure unity of effort (Hansen et ah, 1999). 
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The reason for and benefits of social interaction have been covered, but it is not 
feasible to bring everyone together or send them to conferences. Social interaction can 
be achieved through an information system with similar effects to physical interaction. 
The systems can be particularly successful if they incorporate a method to express the 
need for the information request and feedback to the expert that provides the answer 
(Constant et ah, 1994). 

Data manipulation is not different for information systems designed for 
information sharing than those used for any other reason. However, the ability to 
disseminate, store, send, receive, and process data regardless of physical location is an 
essential element of the information sharing environment. 

3. Organizational Structure 

The structure of an organization establishes the relationships and authorities of 
suborganizations. In doing so, it impacts all other elements of information sharing 
between personnel who are part of the organization. However, because the 
organizational structure is influenced by many other factors, even if information sharing 
is a high priority, the strategy must be flexible for proper integration. The organizational 
structure can create barriers to information sharing, which must be recognized, mitigated, 
and reviewed as it evolves (McLaughlin, 2010). Therefore, planners must understand 
both hierarchical and nonhierarchical organizational structures and how information 
typically flows within them. 

Hierarchical organizations separate areas of expertise in order to process 
information prior to disseminating it to other areas of expertise as input or informing a 
decision maker, see Figure 1. Processing by areas of expertise ensures accuracy and 
streamlines information tracking. Sharing information between suborganizations with 
different areas of expertise is relatively difficult due to both culture and communication 
equipment connectivity. 
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Figure 1. Example of a Hierarchical Organization 

Nonhierarchical organizations integrate areas of expertise into work teams in 
order to “decentralise authority, share information, diffuse and distribute competency” 
(Klenke, 2006, p. 231), see Figure 2. Integration of expertise fosters “innovation, product 
customisation, and technological leverage” (Klenke, 2006, p. 231). Ensuring thorough 
dissemination of corroborated accurate information to all work teams and consensus 
building are challenges in this structure due to both culture and communication 
equipment connectivity. Trust is more significant in nonhierarchical organizations in 
order for consensus building to be effective. 
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Figure 2. Example of a Nonhierarchical Organization 


D, COGNITIVE SYSTEMS ENGINEERING: HOW TO FACILITATE IT 

1, Introduction to Cognitive Systems Engineering 

Cognitive systems engineering is a process used to design computer systems 
based on integrating “the fields of human factors, human-computer interaction, cognitive 
psychology, computer science, artificial intelligence and other related fields” 
(Introduction, n.d., para. 2). One of the approaches used for cognitive systems 
engineering is participatory design, which “places a premium on the active involvement 
of workplace practitioners...in design and decision-making processes” (Participatory 
Design, n.d., para. 1). The participatory design methods of cognitive systems engineering 
integrated for this thesis are critical decision making and contextual inquiry. 

2. Critical Decision Method 

The critical decision method is one of several methods used to conduct a 
cognitive task analysis (CTA). A CTA is used to “analyze and represent the knowledge 
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and cognitive activities workers utilize to perform complex tasks in a work domain” 
(Gordon & Gill, 1997; Schraagen, Chipman, & Shalin, 2000 in Bonaceto & Bums, 2007, 
p. 40). Critical decision making in particular is a “stmctured interview technique 
developed to obtain information about decisions made by practitioners when performing 
their tasks” (Hutchins, Pirolli, & Card, 2007, p. 290). Essentially, the method elicits a 
detailed response from the interviewee by asking him to address an extreme atypical 
situation. By listening to the description and asking additional probing questions, the 
interviewer attempts to tease out the essential elements of the decision making process. 
The decision-making processes of the interviewees are analyzed and used to provide 
input to the systems engineering process. 

3. Contextual Inquiry 

The goal of contextual inquiry is to “support, extend, and positively transform the 
work of individuals, teams, and businesses through computer systems” (Holtzblatt & 
Jones, 1993, p. 177). To reach that goal, it attempts to identify a process for designing 
systems that support people who engage in similar work in many different business 
contexts and cultures ... an expedient process for gathering user information in the time 
available ... a means of gaining appropriate and helpful information about users’ work. 
(Holtzblatt & Jones, 1993, p. 177) 

The contextual inquiry method is particularly useful because of its structure. It 
provides a utilitarian balance of theory and practice. It was conceived to be used with 
other forms of design. The formative principles of context, partnership, and focus, 
“guide our interaction with users regardless of the specific technique or meeting structure 
that we employ” (Holtzblatt & Jones, 1993, p. 179). At the same time, it provides the 
researcher with detailed steps to put the principles into practice through interviews and 
analysis. 

In contrast to the structured interviews of the CTA, contextual inquiry interviews 
occur while observing and interacting with interviewees while they are conducting 
routine tasks. Individual or group interviewers monitor interviewees and engage them in 
conversation throughout the interview. The intent is to observe how the interviewees are 

13 



using the tools provided, record their input, and identify barriers that the users themselves 
overlook. Rather than having a list of structured interview questions, the interviewers 
arrive with a well thought out focus of the interview. As the interview progresses, the 
interviewers adjust their focus to interpret the most realistic view possible of the working 
environment. During the wrap up of the interview, the interviewers confirm that their 
interpretation is accurate (Holtzblatt & Jones, 1993). Multiple rounds of interviews are 
often conducted. 

Analysis of the interviews is best done by a group but can be done by an 
individual when appropriate. The product created through contextual inquiry analysis is 
the affinity diagram. The diagramming process “is a way to synthesize qualitative 
information into conceptual groupings” (Holtzblatt & Jones, 1993, p. 203). It starts by 
refining the focus for analysis and reviewing the interview transcripts in that light. 
During the review process, significant aspects are recorded in detail and capsulated on 
note paper. After reviewing all transcripts, the pieces of note paper are grouped by how 
they relate to each other. Analysts shift note papers between groups until they are 
satisfied that the best relationships are represented. They then name the groups, which 
become level one of the affinity diagram. The analysts group and name the groups two 
more times, resulting in a three-level hierarchy of concepts to guide the engineering of 
the system (Holtzblatt & Jones, 1993). 

E, SUMMARY 

1. Context 

The goal of this thesis is to provide an analysis of information sharing within the 
JTB, which will provide information to inform researchers regarding JTB portal design 
requirements for the follow on theses in the series. In order to analyze information 
sharing, one must study the meaning of information sharing, organizational 
considerations, and relevant methods of cognitive systems engineering that link the 
human and technological aspects. Studying the meaning of information sharing requires 
a common definition and understanding of individual motivation to share. Organizational 
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considerations consist of the organizational culture, environment, and structure. The 
methods of cognitive systems engineering that are relevant to this thesis are CTA and 
contextual inquiry. The way these methods were employed will be covered in the 
following chapter. 
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III. METHODOLOGY 


A, RESEARCH METHODOLOGY 

1. Commencement 

The kickoff meeting for this project provided focus, delineated the scope, and 
recommended the initial research method. The thesis advisors were able to focus the 
student researchers on the mission of this series of theses by providing an introduction to 
the JTB organization, its purpose, the current testing process, and the planned 
contribution of the project. 

This introduction necessarily included organizations other than the JTB in order 
to provide a comprehensive framework that described the entire JIEDDO enterprise for 
the researchers. Follow-on discussions, however, limited the scope for each of the three 
initial theses. Limiting the scope was important because it deconflicted the researchers, 
thus enabling coordination. The scope for this thesis was limited to information sharing 
regarding test execution and reporting for the JTB. Interviewees included (1) individuals 
within the JTB; (2) members of the advanced communications and foreign release 
working groups; and (3) test directors at YPG, APG, and China Lake. Interview 
questions focused on what information each individual needs, expects, receives, fails to 
receive, and provides. 

This researcher was directed to use the critical decision method of cognitive task 
analysis. In accordance with the directive, structured interviews were required. Due to 
the similar scope of the first two theses in the project and practical scheduling and 
logistical requirements, the researchers planned to conduct joint interviews. This 
decision was significant because it positively affected the researchers by enabling support 
during, coordination between, and joint review of the interviews. 


17 



B, 


PREPARATION 


1. Concurrent Processes 

The process of preparing for data collection consisted of the compilation of an 
Institutional Review Board package, selection of participants, and rehearsals. Planning of 
all aspects was conducted concurrently by the researchers, advisors, and sponsor 
representatives in order to streamline the transition from preparation to collection. 

2. Participant Selection 

Selection of participants for data collection for both this thesis and the end-user 
analysis was a coordinated effort between the students, advisors, and sponsor 
representative. The advisors coordinated the interviews at YPG and Naval Air Warfare 
Center Weapons Division Point Mugu (Point Mugu), provided some recommendations 
for other locations, and put the students in touch with the sponsor. Student researchers 
coordinated the interview at China Lake and provided input to the sponsor regarding the 
focus of their topics. The sponsor coordinated the interviews at APC and the JTB in 
Alexandria, Virginia. 

Participants were selected based on both their knowledge and experience with 
JTB testing and availability during the students’ limited travel availability due to class 
schedules. The plan was to include participants with knowledge and experience from 
different perspectives of the testing process. Perspectives of the testers/processors, 
support personnel, and facilitators were all relevant and successfully targeted in the 
selection process. The plan was successfully executed by interviewing. Student travel 
windows were rigid because of travel planning considerations and concurrent class 
schedules. The schedule rigidity resulted in the students’ inability to interview the Acting 
Director and Deputy Director, due to a scheduling conflict. 

3. Rehearsals 

If one is replicating this process, it is prudent at this point to acknowledge the 


benefit of realistic rehearsals. This researcher rehearsed by reading through the questions 
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aloud in order to familiarize himself with the flow and timing of the questions. Such 
rehearsal prepared him to present the questions in a professional manner; it did not 
prepare him for handling unfocused or misdirected answers. This resulted in a lack of 
standardization in analysis because some questions were omitted and others were 
answered from a perspective irrelevant to the focus of the project. For follow on 
researchers, regardless of the amount of time available before collection, at least one live, 
recorded, and timed rehearsal interview with a prepared interviewee is recommended. 
Even if the interviewee is not an expert in the field, the benefits of operationally checking 
all equipment, taking notes, addressing interviewee questions, and facilitating the focus 
of the interview, will improve confidence in oneself and one’s questions, the conduct of 
the actual interviews, and the standardization of the process for more efficient analysis. 

C. INITIAL DATA COLLECTION 
1, Interviews and Surveys 

a. Overview 

The initial data collection phase consisted of 11 structured interviews, 
demographic surveys, and information sharing surveys. The interviews and surveys had 
the same target audience, addressed in Chapter III.A.l. Both surveys were given to the 
interviewees and distributed and collected during the interview session. Locations and 
number of interviews conducted at each location were: YPG (3), China Lake (1), APG 
(1), JTB (4), Point Mugu (1), and NPS (1). All of the student researcher interviews were 
conducted jointly with the end-user analysis researcher. 

There were two overarching factors that affected the interviews. Lirst, 
some questions were omitted, at times purposefully, and at other times, erroneously. 
They were purposely omitted, if they were already answered or clearly not applicable to 
the interviewee. Others, however, were omitted erroneously by this interviewer. The 
second issue was that two of the interviews were not recorded. One interviewee did not 
consent, which was acceptable. The other was not recorded because the interview was 
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conducted in a secure location where recorders were not allowed. The lack of a transcript 
greatly reduced the amount of data collected from that interview. Future researchers are 
advised to ensure that interviews are conducted in an environment where recording is 
allowed, in order to have that option, should the interviewee consent. 

(1) Interviews. The interview format separated the questions into 
five sections: introduction, orientation, information sharing, information flow policy and 
procedure, and conclusion. The text of the questions can be found in tables in the results 
chapter. Only the data collected in the information sharing and information flow policy 
and procedure sections was used for analysis. The questions addressed organizational 
culture, environment, and structure, as well as the aspect of participatory design in 
cognitive engineering. 

The information sharing section of the overall set of interview 
questions consisted of 16 questions that assessed the state of information sharing within 
the organization, identified barriers, and solicited recommendations for improvement. 
Organizational culture norms were addressed in Questions 5-11 and 14. The intent of 
these questions was to document the existing norms for information sharing at multiple 
levels within the organization. The utility of current information systems was addressed 
by inquiring about the media used for communication in Questions 6, 8, and 11. 
Interviewee recommendations solicited in Questions 12, 13, 15, and 16 provided 
participant data for future design considerations. Questions 1-4 did not address specific 
motivational or engineering aspects, however, in establishing a dialogue and eliciting a 
critical decision making situation, they provided an avenue for the interviewee to address 
the aspects most significant to him. 

The information flow policy and procedure section consisted of 
seven questions that addressed leaders’ guidance for sharing information, how 
information was meant to flow, and how those were reflected in practice. In Question 6, 
the normative aspect of organizational culture was addressed using information flow; it 
did not specifically address procedures, which enabled the interviewee to express the 
actual execution of moving information through the organization. Policy is an element of 
belief within organizational culture. Question 4 addressed both the existence and 
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enforcement of an information sharing policy within the JTB. Question 1 addressed the 
interviewee’s perception of the organizational structure relevant to information flow. 
Participatory design aspects were addressed in Questions 2, 3, 5, and 7. 

(2) Surveys. The same demographic and information sharing 
surveys were given to all interviewees. The demographic survey addressed individual 
characteristics, education, experience, and placement within the organization. 

The information sharing survey consisted of 25 questions broken 
into three sections: organizational information sharing (12), intra-organization 
communication structure (5), and organizational culture (8). The information sharing 
section focused on establishing the norms for the organizations and motivation. All of 
the questions except number two addressed organizational norms at the personal level. 
The questions were posed from different perspectives in order to assess not only the 
quantity and quality of sharing, but the balance of giving and receiving as well. Question 
2 addressed motivation. The relatively small amount of attention paid to motivation was 
intentional. The focus in the initial phase was to collect data on what was or was not 
being shared, with a lesser focus on how. The why aspect, motivation, would be 
addressed in the follow-up phase if the initial analysis found a requirement for it. 

The intra-organizational communication section also focused on 
organizational norms, except for a single question on policy. This section differed from 
the information sharing section by taking a slightly broader perspective and addressing 
organizational norms within the interviewee’s suborganization within the JTB. They 
assessed the quality, requirement, procedure and process of the current state of 
information sharing. The policy, or leadership encouragement, was addressed in question 
number four. 

All questions in the organizational culture section addressed norms 
from the broadest perspective within the scope of this thesis by focusing on information 
sharing between the JTB suborganizations 
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b. Initial Data Collection Summary 

The initial round of data collection consisted of 12 interviews at five 
organizations, as detailed in Table 1. The general flow of the interviews included 
introductions, research overview, information sharing interview questions, end-user 
analysis interview questions, and demographic and information sharing survey 
completion. Not all interviewees were asked all of the information sharing interview 
questions, as annotated in the tables presented in the results chapter. 


Table 1. Initial Interviews by Organization 


Organization 

Number of Interviews 

Advanced Communications Working Group 

1 

Joint Improvised Explosive Device Defeat 
Test Board 

5 

Yuma Proving Grounds 

3 

NAWCWD China Lake 

1 

NAWCWD Point Mugu 

1 


D, INITIAL DATA ANALYSIS 

1, Overview 

This section will address the process of analyzing the initial data and how it was 
used in the follow-up phase of data collection. Results of the analysis will be addressed 
in Chapter IV. The initial analysis was conducted following the CTA framework. The 
data was consolidated into narrative that could be further grouped into aspects of the 
cognitive tasks that could, in turn, be targeted for improvement. These steps are similar 
to the cognitive inquiry review process, which enabled the blending of the two 
approaches. 
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2 . 


Consolidation 


Consolidation consisted of transcription of interviews and spreadsheet input for 
qualitative data. Transcription was a shared effort between the researcher and a 
professional service. Of the 11 interviews, the researcher transcribed six, three were 
professionally transcribed, one was not recorded due to security, and the other was not 
recorded due to lack of consent. The researcher’s method of transcription was to listen to 
the cassette tape, or digital media file multiple times and record what was stated in a 
Microsoft Word document. The professional transcription service returned the transcripts 
as Microsoft Word documents as well. For the interviews that were not recorded, the 
researcher typed his notes into another Microsoft Word document. When the 
professional transcripts were received, the researcher consolidated all of the documents 
into a single Microsoft Word document for follow on grouping. 

Surveys consisted of responses that yielded data that could be analyzed both 
qualitatively and quantitatively. Qualitative data was consolidated using the 
consolidation method for unrecorded interviews described above. Quantitative data was 
entered into a spreadsheet for statistical analysis. 

3, Grouping 

Qualitative data was grouped into tables and quantitative data underwent 

statistical analysis. The qualitative data tables extracted and consolidated the most 

significant data from the narrative responses from interviews and surveys. The tables 

were used in the creation of follow-up survey questions and created a pool of notes for 

use in the creation of an affinity diagram in the consolidated analysis phase. The process 

of creating the qualitative data tables consisted of integrating the answers to each 

question, grouping similar responses and identifying trends in the groups. Integrating the 

answers involved assembling the answers to a question under the text of the question 

itself. Then, all of the responses were reviewed together, and similar responses were 

grouped under a single phrase. Here is where the lack of standardization in questioning 

affected the analysis. Because many of the same questions were omitted in multiple 

interviews, those lacking answers sometimes outnumbered those with answers, which 
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reduced their significance. Once the answers were grouped, the groups were reviewed 
for trends and put into tables, which were used in the follow-up and consolidated 
analysis. 

Quantitative data was run through the descriptive statistics application of the data 
analysis add-on. The program returned 15 statistics for each question. The most useful 
statistics for this analysis were the mean, mode, and standard deviation. The statistics 
will be discussed in the following chapter. 

E, FOLLOW-UP METHODOLOGY 

1, Method Determination 

A second round of interviews to target information gaps found in the initial 
analysis was expected as a part of both the cognitive task analysis and contextual inquiry 
methods. The decision to make at this phase was whether to conduct follow-up 
interviews in person or over the telephone. Telephone interviews were chosen due to the 
researcher’s experience, nature of the questions, funding, and time. Because the 
researcher experienced multiple work environments during the initial interviews, 
revisiting them was not deemed necessary. The nature of the questions, based on gaps 
found in the initial analysis, did not require physical collocation with the interviewee. 
Funding may have been made available, if travel was necessary. In this case, however, it 
was decided that travel was not a high enough priority to allocate funds. The amount of 
time required for both coordination and travel would have taken the researcher from 
required classes and slowed the production of this document. Telephone interviews were 
the more efficient in all aspects. 

F, FOLLOW-UP PREPARATION 

1, Participant Selection 

Participants selected for follow-up interviews were either personnel within the 
JTB or contractors who had supported the JTB. Factors that influenced participant 
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selection were the number of relevant positions in the organizations and the participant’s 
perspective on information sharing between the organizations. There are fewer positions 
relevant to this thesis at the ranges than the JTB, due to the nature of the work. At the 
ranges, the majority of the positions focus on the execution of tests at that location, so the 
number of people to interview who have experience with sharing information between 
organizations is limited. The JTB is the opposite. The majority of the positions deal with 
sharing information with other organizations, both inside and outside of the JTB. The 
initial round of interviews adequately covered the relevant positions at organizations 
outside the JTB, so the follow-up interviews were focused on the JTB or contractors with 
experience at the JTB level. The perspective of information sharing at the JTB level 
provided more information on sharing between organizations than individuals. 
Collecting additional data from that perspective was appropriate because it provided 
clearer focus on the scope of this thesis. 

G. FOLLOW-UP DATA COLLECTION 

1, Follow-Up Interviews 

Follow-up interviews consisted of 17 questions divided into five topic areas that 
addressed organizational culture, organizational environment, and participatory design. 
See the tables in the results chapter for the text of the questions. 

Topic 1, specific information that needs to be shared, consisted of four questions. 
Questions 1.2 and 1.3 addressed motivation, and 1.1 and 1.4 addressed participatory 
design. The flow of the questions within Topic 1 identifies a specific type of information 
that needs to be shared and extracts the individual’s motivation to share. Asking the 
interviewee to speculate about why others do not share that information in 1.3 reveals 
more about his motivation to share. A solicitation for recommendations concludes the 
topic area. 

Topic 2, information visibility between organizations, consisted of three 
questions. Questions 2.1 and 2.2 addressed the personal interaction aspect of the 
organizational environment, and 2.3 addressed the norms of the organizational culture. 
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The personal interaction questions targeted the decision making process of finding points 
of contact in order to gain visibility regarding a piece of information in a JTB 
suborganization. Whether or not attempting to gain visibility of information outside of 
the JTB was a norm within the organization was relevant to assess assertiveness within 
the organizational culture. 

Topic 3, organizational structure, consisted of four questions. Question 3.0 
addressed participatory design and the others addressed the organizational environment. 
Identifying the customers from the perspective of the participants provides data to system 
engineers on which organizations or billets to use as a target audience. The remaining 
questions focused on trust within the organizational environment in order to determine 
whether or not it should be targeted as an area for improvement. Trust has added 
significance, due to the JTB being a nonhierarchical organization. 

Topic 4, specific information to be put into a central repository, consisted of five 
questions. Questions 4.1 and 4.2 addressed participatory design and 4.3 through 4.5 
addressed the organizational environment. The participatory design questions solicited 
specific data points on what type of information and content should be included in a 
central repository. The remaining questions dealt with trusting information from that 
repository. These questions differed from the previous topic area by focusing on 
information rather than individuals. 

Topic 5, security classification effects on information sharing, consisted of two 
questions. Both addressed participatory design. Knowing that changing security 
classification was beyond the authority of the JTB, the questions solicited the 
interviewees’ view on what barriers were created by security classification and 
recommendations to mitigate them. 

a. Follow-Up Interview Summary 

The follow-up round consisted of four telephone interviews. The general 
flow of the interviews included introductions, research overview, consent form 
confirmation, survey receipt verification, and interview questions. 
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2 . 


Follow-Up Surveys 


The follow-up surveys were unchanged from the initial round of interviews and 
were administered to all participants. The same content in the surveys was used for 
consistency and because the experienced participants would provide quality input. The 
only change in the survey was the format. Both surveys were altered to have the 
interviewees fill in form fields rather than fill them in manually. The alterations made it 
easier to return to the researcher via e-mail. Consent forms and both surveys were e- 
mailed to all four applicants. All consent forms were signed and returned, but some 
surveys were not returned. Details are included in the tables in the results chapter. 

The difference in the return rate for the initial in person and follow-up e-mailed 
survey responses was significant. Follow-on researchers are advised to make every effort 
to administer surveys in person in order to insure a response. 

H. FOLLOW-UP ANALYSIS 

1. Consolidation and Grouping 

The data for the follow-up interviews was consolidated and grouped using the 
same methodology as the initial interviews. All interview transcriptions were done by the 
researcher. The interview results were grouped independently of the initial interviews 
because the questions were different. The initial and follow-up survey results were 
consolidated because the questions were the same for both rounds of data collection. 

I. CONSOLIDATED ANALYSIS 

1, Affinity Diagram 

The affinity diagram was used to further refine the results of the initial and 
follow-up phases of interviews into a format specifically designed for cognitive systems 
engineering. The complete affinity diagram is presented in the results section. 
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a. Establish a Focus 

The first step in creating the affinity diagram was to establish a focus. The 
primary focus for this affinity diagram was improving information sharing within the 
JTB, as defined for this thesis. The researcher posed the following questions to himself 
throughout the creation process: “What barriers to information sharing exist?”; “How can 
the barriers be mitigated?”; “What recommendations for improvement were provided by 
the users?” 


b. Record Key Aspects 

With the focus questions in mind, the researcher recorded key aspects 
from the initial and follow-up analysis phases on a Microsoft Excel spreadsheet. The 
cognitive task analysis tables created from the interviews and qualitative survey questions 
in the earlier analysis phases provided the key aspects included in the diagram. 

Each key aspect was recorded in a cell on the spreadsheet labeled Tier III. 
The headers were reorganized and categorized on a spreadsheet labeled Tier II. The Tier 
II headers were reorganized and categorized on a spreadsheet labeled Tier I. The 
organization of an example affinity diagram is presented in Eigure 3. 
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Tier I 


Tier II 


Tier III 



Figure 3. Example of an Affinity Diagram 


Although adhesive notes are the recommended material to create an 
affinity diagram, the electronic format was more applicable to this situation because it 
was being recorded by an individual and the results would have to be shared. As an 
individual, using the electronic format was less cumbersome and more mobile than the 
adhesive note method. Also, because the final product will be provided to the follow on 
researcher conducting research on JTB portal design, an electronic format was easier for 
this researcher to reproduce and the follow-on researcher to manipulate. 

c. Tier III 

After all aspects were entered into cells, they were grouped into thirteen 
columns, and each group was given a heading. The headings represented the main points 
in each column, as related to the focus. The headings on Tier III were: Personal 
Relationships, Individual Perceptions, JTB Level Issues, Leadership, Inter-Organizational 
Communication, Best Practices, Standards, Points of Contact, Personnel Turnover, 
Quality of Shared Information, Information Sharing Tools, Access to Information, and 
Information Visibility. 

d. Tier II 

The Tier III headings became the key aspects for two further groupings. 
They were copied and pasted onto another spreadsheet, labeled Tier II. This was done in 
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order to enhance focus solely on the group headings, make the diagram less cumbersome, 
and make the final product more organized and readable. The headings were grouped 
and given their own heading in the same manner as the Tier III aspects. The headings on 
Tier II were: Individual Issues, Leadership Considerations, Management, Information 
Sharing, and Information Availability. 

e. Tier I 

The Tier II headings were copied and pasted onto a spreadsheet labeled 
Tier I, in the same manner as previously described. The Tier II headings were then 
grouped and given another heading. The Tier I headings were the top-level headings for 
the affinity diagram. The Tier I headings were: Individuals, Leadership, and Information. 

f Record the Diagram 

Recording the diagram was not an issue, due to the use of the electronic 

format. 


2. Surveys 

a. Qualitative Data 

The qualitative data from the initial and follow-up interview question 
analysis was included in Tier III of the affinity diagram. 

b. Quantitative Data 

The quantitative data from the initial and follow-up information sharing 
survey analysis was consolidated and run through the descriptive statistics application of 
the data analysis add-on. The program returned 15 statistics for each question. The most 
useful for this analysis were the mean, mode, and standard deviation. The consolidated 
statistics will be discussed in the following chapter. 
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J. SUMMARY 

1. Overview 

The research methods for cognitive task analysis and contextual inquiry were 
incorporated for this thesis. Data collection consisted of structured interviews, a 
demographic survey, and an information sharing survey. Questions were focused on 
individual motivation, organizational influences, and cognitive engineering. Data 
analysis included the creation of data tables, in accordance with cognitive task analysis, 
an affinity diagram, in accordance with contextual inquiry, and statistical analysis. 

2. Data Collection 

a. Interviews 

Data collection consisted of structured interviews and surveys. The 
interviews were conducted in two rounds, initial and follow-up, each with different 
questions. The initial questions were conducted in person at YPG, China Lake, APG, 
JTB offices. Point Mugu, and NPS. Follow-up interviews were conducted via telephone 
with JTB members and contractors with JTB level experience. 

b. Surveys 

Two surveys were administered, a demographic survey, and an 
information sharing survey. Both were administered in two rounds, initial and follow-up, 
with the same content, to the same people that were interviewed. The initial round was 
administered in person and all responses were gathered. The follow-up round was 
administered via e-mail and multiple responses were not returned. 
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3. 


Data Analysis 


a. Qualitative 

The qualitative information from each round of interviews and the 
qualitative questions from the surveys were used to create tables that grouped the data 
into cohesive data points that revealed the cognitive nature of the tasks involved. 
Separate tables were created for each round. Those data points were then consolidated 
and used as the key aspects in the creation of a three-tier affinity diagram, which will be 
provided to a follow on researcher conducting research on JTB portal design. 

b. Quantitative 

Quantitative information from surveys from each round was run through 
statistical analysis software separately. The data was then consolidated and run through 
the same software. The initial, follow-up, and consolidated statistics will be discussed in 
the following chapter. 
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IV. RESULTS 


A, OVERVIEW OF RESULTS 

1. Presentation of Results 

This chapter presents the results of the research. The methodologies used were 
designed to produce tabular results, which create the majority of the chapter. Text is used 
minimally to provide context for tabular content or present results that are not suitable for 
tabular presentation. 

2. Consolidated Demographics 

The demographic information for the 15 interviewees shows that they are well 
educated and experienced, especially considering the relatively short existence of the 
JTB. It also shows the diversity in duties among the members of the organization, which 
is reflected in the interview results as well. Due to the number of demographic responses 
that were not appropriate for tabular presentation, the results will be described in the 
following paragraphs. 

The mean age group of the participants was 31 to 40 years old. Fourteen of the 
15 participants were male. The education level of the participants ranged from high 
school to doctorate degree, with the majority of the participants having a bachelor’s 
degree. Seven of 15 had taken a single formal course relative to C-IED testing, and one 
had taken several related courses. 

Twelve of 15 participants had testing experience. The three participants without 
testing experience were JTB employees not directly involved in testing. Of those with 
experience, the average length of experience with C-IED testing was 4.8 years. Eive 
participants were in positions directly involved with testing, and 10 were not. The 
average time in their current position was one to five years, one had less than one year 
and two had three- to ten-year’s experience. 
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Regarding their place of employment, six were from the JTB, three were involved 
in testing at YPG, one was from the Advanced Communications working group, one was 
from China Lake, one was from a program office, and three were not associated with the 
JTB. 

Five of the 15 had worked as members of multiple JTB organizations. One at the 
JTB had served on the National Assessment Group and as a technical advisor, one from a 
test range listed another position at the same range, one in the Advanced 
Communications Working Group had experience at a test range, one at a program office 
listed another C-IED related program office, and one provided no further details. Of the 
15, seven had not participated in any JTB sponsored tests, two participated in less than 
10, two participated in between 90 and 100, and four participated in 200 or more tests. 

3. Interview Summary 

a. Initial Interview Summary 

The initial interview questions were separated in two sections, reflected in 
the two tables that follow. The tables include the question number, question text, a 
summarized analysis of the answers, and the number of responses for each question. 
Detailed data points from the interviews will be presented in the CTA tables and the 
affinity diagram later in this chapter. 

Table 2 presents responses to the survey questions in the Information 
Sharing Section, which focused on the individuals’ experiences with sharing information. 


Table 2. Initial Interview Information Sharing Section Summary 


# 

Question Text 

Answer Summary 

N* 

1 

From your perspective, what 
differences exist regarding 
information needs between the 
organizations? 

Information needs are similar among the test facilities. 
However, organizations that process and distribute the 
results have different requirements. 

11 

2 

Please describe a time when you 
needed to use information from 
another organization. 

All examples ended in receiving the required information. 
Only one dealt with another organization within the JTB, 
and that was contingent on a personal relationship. 

5 
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3 

Please describe a time when you 
were asked to share your 
information with another 
organization. 

Examples included inter-testing facility sharing 

5 

4 

Please describe an exceptionally 
challenging experience when you 
successfully or unsuccessfully 
shared information with the JTB or 
one of its organizations. 

Positive feedback focused on cooperation between test 
ranges. Negative feedback referred to information 
visibility and availability, changing requirements, getting 
raw data from a test facility, and sharing with coalition 
partners. 

10 

5 

In your experience with JTB tests, 
to what extent does your 
organization provide information to 
other organizations? 

Information is provided to others through personal points 
of contact and by posting it to the JTB portal and Theater 
Support Web Tool (TSWT). Answers did not explicitly 
address the extent of providing information. 

10 

6 

When was the last time you or your 
team provided information to 
another organization? How did you 
send that information? Is that 
typical? What is typical? 

Information provision ranged from constantly to semi¬ 
annually, with a majority being daily. All means were 
typical and included personal communication, e-mail, 

JTB Portal, video teleconference, and telephone. 

11 

7 

In your experience with JTB tests, 
to what extent does your 
organization receive information 
from other organizations? 

Information reception ranged from none to ‘as it is 
requested’, with a majority being ‘as it is requested’. A 
single response included receiving information without 
requesting it, and that dealt with budgeting. 

11 

8 

When was the last time you or your 
team used information received 
from another organization? How 
did you receive that information? Is 
that typical? What is typical? 

Timeframes ranged from currently to a couple of months 
ago. Noted reception methods included e-mail, video 
teleconference, and the JTB database. 

11 

9 

In your experience, how much of 
the information you needed from 
other organizations have you been 
able to obtain? To the extent that 
you were unable to obtain all of the 
information you needed, what was 
the primary cause of the problem? 

Information obtain ability ranged between ‘a fair amount’ 
to all of it, with a single exception the replied ‘no’. 
Problems consisted of a lack of timeliness, lack of access 
to the information, managerial resistance, and reception 
being based on inconsistent personal relationships 
because the organization is always changing. 

11 

10 

What are the most significant 
aspects, positive or negative, of 
sharing information between JTB 
organizations? 

Positive: ranges working together; sharing once makes it 
easier the second time; and everything is done for the war 
fighter. Negative: people don’t know who to ask for what 
type of information; people at different locations do not 
know each other; and the receiver must understand the 
information before it should be shared. 

6 

11 

Does your organization share 
information with other 
organizations through ad hoc 
means or means other than those 
provided through the JTB? If yes, 
what are they? How well do you 
think they would work if they were 
used throughout the JTB and other 
organizations? 

No ad hoc methods were provided. Means provided 
included working groups, video teleconferences, the JTB 
Portal, and test protocols. 

6 
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12 

Exchanging more information with 
which other JTB organization 
would improve your organization’s 
performance (examples if needed; 
test range, working group, board 
members)? 

The majority of responses did not answer the question, 
but were addressed in the analysis. Relevant responses 
were other test ranges, JIEDDO, and JEIDDO modeling 
and simulation. 

11 

13 

What changes would you make to 
improve information sharing 
between all personnel involved in 
JTB testing? 

Recommendations related to the following: a better 
database, ensuring awareness of available information, 
ensure the principle players are coordinating with each 
other, disseminate working group information, and 
disseminate an up-to-date organizational chart. 

11 

14 

What, if any, impediments to 
sharing test results are you aware 
of? 

Impediments consisted of: corporate culture, financial 
competitiveness of the ranges, inability to access other 
range databases, approval required before information can 
be released (one location), results from other customer 
tests are not shared, and foreign release ability 

10 

15 

Please describe any information 
gaps that exist regarding 
information sharing and how you 
would fill them 

Eour of the eight responses noted the absence of gaps. 
Others made the following recommendations: create an 
information czar to manage working group information; 
make PMS-408 the Joint Counter Radio-Controlled 
Electronic Warfare Program Manager; and standardize 
data collection and storage (file formats, etc.). 

8 

16 

Would more sharing of information 
between organizations help or 
hinder timely and accurate results? 
How? 

All responded that it would help, but none described how. 

2 


*Number of responses out of 11 


Table 3 presents the results of the participant responses to items in the 
Policy and Procedure Section, which focused on sharing between organizations and the 
guidance from the top of the hierarchy. 


Table 3. Initial Interview Policy and Procedure Section Summary 


# 

Question Text 

Answer Summary 

N* 

1 

How does the information that you 
process or create get to the JTB? 

Responses consisted of the following: it depends on the 
personal relationships involved, directly to the JTB, post 
to the JTB portal, directly to foreign release officer, post 
to Theater Support Web Tool (TSWT), send it through a 
working group 

6 

2 

What specifically enables or inhibits 
information sharing with the JTB 
and other organizations? 

Culture, personal bias, nothing, lack of visibility on 
what’s been collected, security classification 

5 

3 

What procedural improvements do 
you recommend to mitigate any 
inhibitions to information sharing? 

Recommendations consisted of the following: bring all C- 
lED testing under the JTB’s purview, current process is 
good, (oracle) database will help, all data from all 
locations should flow to a backup site, create standard 
operating procedure for information sharing, and 1 don’t 

10 
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know 


4 

Is sharing information between 
organizations and the JTB supported 
by clear policy and guidance? In 
your experience, is this policy 
routinely enforced? 

Responses included: guidance is there, guidance is not 
there, there is a policy, there is probably a policy - but I 
don’t know what it is, there is not policy, and people 
follow norms rather than written policy 

11 

5 

What policy changes, if any, would 
you recommend to enhance 
information sharing between 
organizations and the JTB? 

Recommendations were: have ranges work together on 
tests, create a database with the ranges and their 
customers, all JTB members should have to visit the test 
ranges to establish personal contacts, create turnover 
binders, and enforce existing policy 

10 

6 

Please explain any uncertainty or 
confusion regarding information 
flow into or out of your 
organization. In your opinion, how 
can the uncertainty or confusion be 
mitigated? 

Of those who responded that there is uncertainty, it is due 
to: test ranges have other customers besides the JTB, 
personnel turnover at JTB requires the establishment of 
new personal relationships, and people don’t know who to 
go to for what information. 

Recommendations for mitigation included: process maps 
for data flow processes, infrastructure, research & 
development, get employees together to build personal 
relationships, and creating an information sharing policy 

10 

7 

Please describe what you think 
would be the most efficient route 
and method to exchange 
information between organizations. 

Recommendations included: add a pop-up alert for new 
information on the JTB portal and implement a quality 
easy to use web interface 

3 


*Number of responses out of 11 


b. Follow-Up Interview Summary 

The follow-up interview format was separated into five topics, reflected in 
the following five tables. Topic one questions addressed specific information that needs 
to be shared. The questions under this topic were meant to clarify preliminary analysis of 
the initial survey results, which showed that interviewees were receiving all of the 
information that they needed, but thought more sharing would be beneficial. While many 
good data points were raised and included in the CTA and affinity diagram, not all of the 
specific details that were targeted by the questions were provided. 


Table 4. Follow-Up Interview Summary: Specific Information That Needs to be 

Shared 


Topic 1 - Specific Information that Needs to be Shared 

# 

Qnestion Text 

Answer Snmmary 

N* 

1 

What specific type of information 
needs to be shared? 

Most of the responses did not answer the question, but 
were included in the analysis. The one response that did 

4 
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answer the question recommended that data being 
collected by the program offices should be shared with 
the JTB and the field. 


2 

Considering your answer to the 
previous question, why are you not 
giving that specific information? 

Responses were either that the question did not apply 
because the interviewees were not sharing the information 
or did not directly answer the questions, but were 
included in the overall analysis. 

4 

3 

Regarding that same information, 
why do you think that you are not 
receiving it? 

Suggestions included; a lack of trust, the difficulties 
involved, the lack of a process, and some people don’t 
perceive a requirement to share 

4 

4 

How can the situation be remedied 
to ensure people are able to share 
the right information? 

Recommendations included; post information to a single 
location to ease access, ensure visibility of information, 
increase coordination, create a better process for sharing, 
and conduct working groups to integrate the methods of 
all locations. 

4 

5 

Who are the JTB’s customers? 

DoD, operating forces, coalition partners 

4 


*Number of responses out of four 


Topic two questions addressed information visibility between 
organizations. Issues raised during initial interviews included of a reliance on personal 
relationships for sharing information, problems being able to figure out who to contact to 
get information, and issues with visibility of who has what information. The visibility 
issue referred to both within the JTB and outside, including program managers and other 
range customers. Therefore, questions related to this topic focused on how interviewees 
used existing points of contact and how they created new contacts in order to discover 
information that was not readily visible and accessible. 


Table 5. Follow-Up Interview Summary: Information Visibility Between 

Organizations 


Topic 2 - Information Visibility Between Organizations 

# 

Qnestion Text 

Answer Snmmary 

N* 

1 

When you need a piece of 
information, please describe the 
process you use to decide who to 
ask for it? 

Processes included; use the organizational chart, use the 
JTB portal to find the information without asking 
someone, and weekly video teleconferences 

4 

2 

If you have a need for information, 
but do not have a personal 
relationship with someone who has 
it, how do you find out who to 
contact to get the information you 
need? 

The answers were the same as question 1. 

4 

3 

Understanding that organizations 
outside of the JTB can have 

Three of the four answered yes, while the other answered 
that it should be taken care of through the contracting 

4 
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proprietary information, if you had 

process. Means of finding points of contact consisted of 



reason to believe that someone 

networking through working groups and at conferences, 



outside of the JTB, such as a 

getting them from the chain of command, and calling a 



program manager or civilian 

business contact number and talking to people within the 



business, had information that could 
benefit your organization, would 
you contact them? 

organization. 



*Number of responses out of four 


Topic three questions addressed organizational structure. The JTB does 
not fit the mold of a hierarchical organization, due to the geographically separated nature 
of the enterprise, which includes a collection of different services and agencies 
conducting different, but similar tasks. As a result, the JTB is partially a flat 
organization. In order for flat organizations to work, the employees must trust each other 
and/or the information the other provides. The questions under this topic focused on 
trust. 

Table 6. Follow-Up Interview Summary: Organizational Structure 


Topic 3 - Organizational Strnctnre 

# 

Qnestion Text 

Answer Snmmary 

N* 

1 

Do you trust the information that 
you receive, whether or not you 
have met the person sending it? 

The majority said yes, but the others would check the 
sender’s reputation with others in the same field. 

4 

2 

Do you trust other members of the 
JTB organization enough to share 
information with them, even if you 
have never met them face to face? 

The overall consensus was yes, if they had a need to 
know. 

4 

3 

Do you consider other members of 
the JTB organization experts in their 
particular fields? 

Half would and the other half would want to confirm their 
credibility before assuming they were experts. 

4 


*Number of responses out of four 


Topic four questions addressed specific information to be put into a 
central repository. The purpose of this topic was to elicit specifics regarding the content 
and use of a central repository because the creation of ‘a database’ was a recurring 
recommendation by multiple interviewees for many topics in the initial round of 
interviews. 
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Table 7. Follow-Up Interview Summary: Specific Information to be Put Into a 

Central Repository 


Topic 4 - Specific Information to be Pnt into a Central Repository 

# 

Qnestion Text 

Answer Snmmary 

N* 

1 

What, specifically, should be in a 
central information repository for 
the JTB? 

Data that is appropriate for wide distribution, including: 
test plans, test reports, scenario data, 
conditions/environmental, performance data, and 
representation of the sensor for instrumentation 

4 

2 

Should the repository consist of 
both technical and administrative 
information (scheduling, budgeting, 
etc)? 

Responses were evenly split between making it a one-stop 
shop for all information and making it a strictly technical 
store of knowledge 

4 

3 

Do you believe that others would 
use your information correctly? 

Why? 

All agreed that the information would be used correctly. 
Some additional information included the need for a 
defined organization for the information and feedback 
mechanisms on the JTB portal. 

4 

4 

Do you trust others to use your 
information? Why? 

Three of the four stated yes, and the other noted that it 
had to be made available and that most people would use 
it correctly 

4 

5 

Would you trust the information 
others entered into the central 
repository? Why? 

All agreed that they would trust the information. Access 
controls provided additional confidence for one responder 

4 


*Number of Responses out of four 


Topic five questions addressed the effects of security classification on 
information sharing. Multiple initial interview responses stated that security 
classification inhibited information sharing, so the questions under this topic were meant 
to provide clarification. 


Table 8. Follow-Up Interview Summary: Security Classification Effects on 

Information Sharing 


Topic 5 - Security Classification Effects on Information Sharing 

# 

Question Text 

Answer Summary 

N* 

1 

What makes the security classification of 
a report an inhibition to information 
sharing? 

Security requirements are there for a reason and must be followed 
regardless of the situation. If a report is to be shared with a 
coalition partner, at least two different copies must be processed. 

4 

2 

What are your suggestions to resolve the 
issue? 

If multiple security versions will be required, prepare them 
simultaneously from the start, rather than editing the finished 
product 

4 


*Number of Responses out of four 
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B, QUALITATIVE RESULTS 


1. Cognitive Task Analysis 

a. CTA Table Description 

This section describes which data points were included in the CTA, how 
they were organized and presented, and acknowledges the absence of data points in 
specific areas. The CTA tables include data points from initial interviews, follow-up 
interviews, and qualitative answers to survey questions. 

The data points were initially organized by whether they identified barriers 
or enablers to information sharing, or recommendations to improve information sharing. 
Those data point groupings were then subdivided into the motivational and organizational 
culture, environment, and structure influences described in the methodology chapter. The 
result was a collection of twelve tables that organized the data points by cause (the 
methodology chapter organization), and effect (barrier, enabler, or recommendation). 

The data points did not address all subdivisions, so some of the columns 
have no entries, and are annotated accordingly. Also, three tables had no data points, so 
they are not included; they were: organizational structure enablers, motivational 
recommendations, and organizational structure recommendations. 

Table 9 presents the motivational barriers based on information collected 
from participant surveys. It includes participant responses related to intrinsic, intrinsic 
hedonic, and extrinsic motivation. 


Table 9. Motivational Barriers 


Intrinsic 

Intrinsic Hedonic 

Extrinsic 

Meetings that include coalition 
partners restrict some 
communications 

Maintain organizational 
reputation by preventing misuse 
of data by other organizations 

Everyone trying to secure 
funding 

People are reserved in large video 
teleconferences because they are 
not willing to share with 
information with all of the 
participants 

Maintain organizational 
reputation by the provider 
restricting sharing to only those 
who he believes understand what 
to do with the information 

Sharing information is helping 
your competition, which puts 
your job at risk 
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Individuals hoard information to 
make themselves seem more 
powerful 

Testers and range workers fear 
losing their jobs 


Individuals don't share in order to 
avoid being judged by their peers. 



The belief that the info is mine so 
no one else can interpret it 



Individual's fear of data being 
misused 



Protect the war fighter by 
ensuring that information is not 
taken out of context 



Table 10 presents the organizational culture barriers based on information 
collected from participant surveys. It includes participant responses related to beliefs and 
norms. Belief data points dealt with issues related to policy and norm data points dealt 
with issues related to process. 


Table 10. Organizational Culture Barriers 


Beliefs (Relating to Policy) 

Norms (Relating to Process) 

Managerial resistance to obtaining data from 
another organization 

There is no process in place to share information 

Institutionalized financial resource competition 
between test ranges stops information sharing 

Most tests are set up as individual tests to be 
conducted at a single range. 

Test ranges have different priorities than the JTB 
because they have to work with other customers to 
obtain additional financial resources 

Some organizations attempt to repress certain 
information if it might be harmful to a system 

Rivalries between sites and services create barriers 

Lack of visibility on updated documents 

Bureaucracy has made the organization ineffective 

Inability to locate documents that are known to exist 
in other organizations 

Management is inadequate at multiple levels 

Falsely assuming that others have access to the 
same information 

Decisions are made in a risk averse environment 

Individuals don’t know who has what information 

Belief throughout the organization that sharing 
information may result in losing business to another 
organization 

Obtaining raw unfiltered data from a test range is a 
challenge 

No commonality of a strategic plan between ranges 
or from higher 

Requirements continually change 

There is no standard method or format for 
requesting, collecting, storing, or disseminating data 

Information received is not normally timely or 
accurate, which leads to getting the information 
through different means 

There is are no formal procedures for information 
sharing 

When sharing occurs, the amount of information 
received is inadequate 

The lack of a written Information Sharing policy 
results in confusion through reliance on a set of 
unwritten norms that individuals interpret 
differently 

People get the information they need only after 
working through barriers 
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No common data schema or data collection formats 


Requirements levied on individuals change 
frequently 

Sometimes an ultimatum is necessary, even if others 
know the information is required 

The corporate culture focuses on individuals gaining 
power over others 

A certain test range will share only complete test 
reports officially approved for release 

The corporate culture does not value information 
sharing 

Much of the information gathered is proprietary and 
not shared 

Individuals don’t see themselves as part of an 
enterprise 

There is a lack of access to information from other 
facilities and ranges 


There is a lack of access to documents that are 
known to be available to others 


Table 11 presents the organizational environment barriers based on 
information collected from participant surveys. These include participant responses 
related to personal interaction, information systems, and incentives. 


Table 11. Organizational Environment Barriers 


Personal Interaction 

Information Systems 

Incentive 

Sharing is often contingent upon 
who you know 

Theater Support Web Tool is not 
user friendly 

Nothing Noted 

People don’t know all the 
individuals and personalities, so 
they have a difficult time 
receiving the information they 
need 

The enterprise lacks a good 
central repository that has all the 
metadata and the data required to 
make use of the information 


Some individuals share only what 
is requested, despite knowing 
more is required 

There are issues with permissions 
in the information system 


Working group members don’t 
disseminate what they learn with 
co-workers 

Not being able to access other 
ranges database for information. 


Sharing is based on the 'good ole 
boy' system 



Individual biases 



Lack of trust throughout the 
organization 



The reduced number of working 
groups results in less personal 
interaction 



High rate of personnel turnover 



Significant turnover in JTB 
leadership 



Can’t figure out who to talk to for 
certain information 



Information is generally 
requested from unknown points 
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of contact 



People don’t know who to ask for 
what information 



Some individuals don’t know 
there is a necessity to share 




Table 12 presents the organizational structure barriers based on 
information collected from participant surveys. It includes participant responses related 
to hierarchical and nonhierarchical organizations. 


Table 12. Organizational Structure Barriers 


Hierarchical 

Nonhierarchical 

Organizational barriers exist between services 

JTB organization is constantly changing 

Organizations are managed as "stovepipes" 

Geographical distribution of JTB organizations 
challenges information sharing 


The JTB does not have sole authority over is 
component organizations, which results in a lack of 
unity of leadership and purpose 


Table 13 presents the motivational enablers based on information 
collected from participant surveys. It includes participant responses related to intrinsic, 
intrinsic hedonic, and extrinsic motivation. 


Table 13. Motivational Enablers 


Intrinsic 

Intrinsic Hedonic 

Extrinsic 

Sharing information to improve 
everyone's performance. It 
doesn't matter who takes the 
credit, as long as the job is done 
right 

Everything is done for the war 
fighters, and they are appreciative 
of it 

Information required for 
budgeting is received quickly 

Share to clarify information 

Everyone has emotional ties to 
making sure the war fighters are 
safe 


Share because it is required to get 
the job done 

Individuals will use information 
in the central repository correctly 
if their interest is to help the guys 
overseas. 


Share so everyone can benefit 
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Table 14 presents the organizational culture enablers based on information 
collected from participant surveys. It includes participant responses related to beliefs and 
norms. 


Table 14. Organizational Culture Enablers 


Beliefs (Relating to Policy) 

Norms (Relating to Process) 

Creating standard operating procedures within 
departments 

Multiple ranges cooperating on a single test 

Organizational culture based on reaction to 
requirements from the war fighter means inherent 
flexibility 

Analysis metrics and test design information are 
often shared between organizations 

Creating an environment that is open to sharing and 
protects proprietary interests 

Sharing information with modeling and simulation 
may lead to better models and less open air testing 

Guidance from higher leadership 

Protocols are open to interpretation 

Internal review by JIEDDO 

Some commonality of requirements between test 
ranges 

The new deputy director's primary goal has been to 
improve communication 

Working Groups 


Most people get the amount of information they 
need 


Weekly status reports are done and recorded 


Culture is better now than it was two or three years 
ago 


Table 15 presents the organizational environment enablers based on 
information collected from participant surveys. It includes participant responses related 
to personal interaction, information systems, and incentives. 


Table 15. Organizational Environment Enablers 


Personal Interaction 

Information Systems 

Incentive 

Personal relationships lead to 
points of contact 

Transitioning to a new Oracle 
database 

Nothing Noted 

Getting information depends on 
personal relationships 

The JTB is creating a common 
server environment that will 
attach people at all locations 


Personal relationships get assets 

Establishing a threat database to 
track threat resources and their 

status 


Personal relationships get access 
to information 

JTB portal home page designed 
as a wheel for easy access to all 
aspects of the organization 
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Attending conferences to develop 
personal relationships and 
establish points of contact 

Read Me files are available for 
everything on the JTB portal 


Working groups are a good start 
to communicating between ranges 

Access controls on who can post 
to the JTB portal increase trust in 
the posted information 


Physical collocation of leadership 
enables better communication 

Points of contact for the 

Knowledge Integration and 
Networking Group are on the top 
of the page of the JTB Portal 


Weekly meetings 

High quality hyperlinks from the 
JTB portal 


Secure Video Teleconference 

Test result availability on the 
Theater Support Web Tool 


Conferences 

Redundant posting on the Theater 
Support Web Tool and JTB Portal 


Physical collocation of testers 
from multiple ranges 

Tying together IT infrastructure 


Face-to-face groups enhance 
sharing 

Theater Support Web Tool 


Sharing establishes relationships 
that make it easier to share again 



Getting together to solve 
problems 

JTB portal 



Email (NIPR and SIPR) 



Telephone 



Table 16 presents the organizational culture recommendations based on 
information collected from participant surveys. It includes participant responses related 
to beliefs and norms. 


Table 16. Organizational Culture Recommendations 


Beliefs (Relating to Policy) 

Norms (Relating to Process) 

Change the culture to a single unified team 

Restrict access of information to individuals who 
know what to do with the information so everyone 
is willing to share 

JTB should provide cleaner roles for testing 
organizations 

Make sure everyone is aware of all available 
resources 

Senior Leadership should enact a new management 
model across all agencies, at all levels of seniority 

Working groups should do a better job of letting 
others know what information they have to share 

Create a position to consolidate and distribute 
information and updates from working groups 

Continue to have projects where different test 
ranges work together 

Standardize the file format of the most commonly 
used test data 

Create procedures for information to flow from all 
locations to a central repository 

Formalize the information sharing processes 

Establish the processes for coordination 

Create a written set of guidance for continuity for 

Create process maps for data flow processes. 
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each position in order to lessen the impact of 
personnel turnover 

infrastructure, and research & development 

Create an information sharing policy for the JTB 

Data being collected by the program office needs to 
be shared with the JTB 

Develop overarching protocols 


Build trust with JIEDDO modeling and simulation 



Table 17 presents the organizational environment recommendations based 
on information collected from participant surveys. It includes participant responses 
related to personal interaction, information systems, and incentives. 


Table 17. Organizational Environment Recommendations 


Personal Interaction 

Information Systems 

Incentive 

Reinstate JTB run range worker 
meetings 

The JTB portal should include; 
climate data, atmospheric 
conditions, time, space, and 
position information 

Nothing Noted 

Create a collective situational 
awareness for the organization 

A central repository should 
include: scenario data, 
environmental conditions, 
performance data, representation 
of the sensor for instrumentation, 
test plans, and test reports that 
constitute every data element or 
data record 


Get the JTB leadership together 
to make sure that everyone is on 
the same page 

Create a multi-security-level 
database that links data within 
reports at different levels 


JTB employees should visit all of 
the organizations to understand 
how they work 

Add a selectable pop-up notice 
within the JTB portal 


Conduct an offsite to give people 
the chance to get to know each 
other 

An easy to use well implemented 
web interface 


Publish a complete organizational 
chart that includes individuals, 
their expertise, and contact 
information 

Create a common data store 
between the JTB and JIEDDO 


JTB needs to establish points of 
contact at each test location that 
can help to get people the right 
test data that they need to be 
sharing 

Create a central data repository 
where everyone collects, shares, 
and presents data in a similar 
fashion 


Create and disseminate a point of 
contact list for each organization 

All data in a central repository 
should be appropriate for wide 
distribution 


Increase telephone discussions 
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2, Affinity Diagram 


a. Background 

This section will establish the focus and explain the presentation of the 
affinity diagram. As stated in the methodology chapter, the primary focus for this affinity 
diagram was to improve information sharing within the JTB, as defined for this thesis. 
With that in mind, the secondary integrated and supporting focus was to collect data that 
could be used by follow-on researchers in the creation of a portal. The three tiers 
represent different levels of abstraction in order to enable the user to focus on the most 
pertinent level. 

Due to the quantity of data points in Tier III of the diagram, some 
explanation of how to read the tables is necessary. First, as described in the previous 
chapter, the diagram is built from the bottom up, so Tier III contains all of the qualitative 
data points from all interviews and surveys. There are 13 columns in Tier III, which is 
presented in sections for readability. As a result, the descriptions before each Tier III 
table refer to column numbers, which represent the original organization of columns from 
left to right. Column order is significant because the left to right organization was 
correlated between all three tiers of the diagram. Figure 4 presents a shell of the affinity 
diagram in order to clarify how the sectioned tables of Tier III incorporate into the 
diagram as a whole. 


Tieil 


Doa 

Tiein 


□ n n 

Tierlll 

dtiot 

□QoDon□□□□ 


Figure 4. Shell of the Affinity Diagram 
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b. Affinity Diagram Results 


The results of the affinity diagram are presented in the tables that follow. 
Table 18 presents the first and second columns of Tier III. The table presents 
considerations relevant to personal relationships and individual perceptions in the design 
of an information system for the JTB. 


Table 18. Tier III Diagram: Personal Relationships and Individual Perceptions 


Personal Relationships 

Individual Perceptions 

The corporate culture focuses on individuals gaining 
power over others 

Individuals don’t see themselves as part of an 
enterprise 

Sharing is often contingent upon who you know 

Belief throughout the organization that sharing 
information may result in losing business to another 
organization 

Personal relationships lead to points of contact 

Maintain organizational reputation by the provider 
restricting sharing to only those who he believes 
understand what to do with the information 

Individuals hoard information to make themselves 
seem more powerful 

Sharing information is helping your competition, 
which puts your job at risk 

Personal relationships get assets 

Testers and range workers fear losing their jobs 

Sharing is based on the 'good ole boy' system 

Maintain organizational reputation by preventing 
misuse of data by other organizations 

Personal relationships get access to information 

Lack of trust throughout the organization 

People don’t know all the individuals and 
personalities, so they have a difficult time receiving 
the information they need 

Individuals don't share in order to avoid being 
judged by their peers. 

Getting information depends on personal 
relationships 

Share because it is required to get the job done 

Individual biases 

Share to clarify information 

Information is generally requested from unknown 
points of contact 

The belief that the info is mine so no one else can 
interpret it 

Sometimes an ultimatum is necessary, even if others 
know the information is required 

Individuals will use information in the central 
repository correctly if their interest is to help the 
guys overseas. 

Some individuals share only what is requested, 
despite knowing more is required 

Sharing information to improve everyone's 
performance. It doesn't matter who takes the credit, 
as long as the job is done right 

Everyone trying to secure funding 

Share so everyone can benefit 

Attending conferences to develop personal 
relationships and establish points of contact 

Individual's fear of data being misused 

Conferences 

Everything is done for the war fighters, and they are 
appreciative of it 

Working group networking 

Everyone has emotional ties to making sure the war 
fighters are safe 

Conduct an offsite to give people the chance to get 
to know each other 

Protect the war fighter by ensuring that information 
is not taken out of context 

Points of contact for the Knowledge Integration and 
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Networking Group are on the top of the page of the 
JTB Portal 


Sharing establishes relationships that make it easier 
to share again 



Table 19 presents the third and fourth columns of Tier III. The table 
presents considerations relevant to JTB level issues and leadership in the design of an 
information system for the JTB. 


Table 19. Tier III Diagram: JTB Level Issues and Leadership 


JTB Level Issues 

Leadership 

Organizational culture based on reaction to 
requirements from the war fighter means inherent 
flexibility 

The corporate culture does not value information 
sharing 

The JTB does not have sole authority over its 
component organizations, which results in a lack of 
unity of leadership and purpose 

Culture is better now than it was two or three years 
ago 

Institutionalized financial resource competition 
between test ranges stops information sharing 

Change the culture to a single unified team 

Physical collocation of leadership enables better 
communication 

The new deputy director's primary goal has been to 
improve communication 

Most tests are set up as individual tests to be 
conducted at a single range. 

Senior leadership should enact a new management 
model across all agencies, at all levels of seniority 

Create a position to consolidate and distribute 
information and updates from working groups 

Management is inadequate at multiple levels 

Bureaucracy has made the organization ineffective 

A certain test range will share only complete test 
reports officially approved for release 

Organizations are managed as "stovepipes" 

Managerial resistance to obtaining data from 
another organization 

JTB organization is constantly changing 

JTB should provide cleaner roles for testing 
organizations 

The reduced number of working groups results in 
less personal interaction 

Requirements continually change 

Geographical distribution of JTB organizations 
challenges information sharing 

Requirements levied on individuals change 
frequently 


Management intervention is often necessary to 
require others to share information 


Some individuals don’t perceive a necessity to share 


Create a collective situational awareness for the 
organization 


Get the JTB leadership together to make sure that 
everyone is on the same page 


Creating an environment that is open to sharing and 
protects proprietary interests 


Decisions are made in a risk averse environment 


Guidance from higher leadership 
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Table 20 presents the fifth and sixth columns of Tier III. The table 
presents considerations relevant to inter-organizational communications and best 
practices for information sharing in the design of an information system for the JTB. 


Table 20. Tier III Diagram: Inter-Organizational Communication and Best Practices 


Inter-Organizational Commnnication 

Best Practices 

Reinstate JTB run range worker meetings 

Continue to have projects where different test 
ranges work together 

Data being collected by the program office needs to 
be shared with the JTB 

Physical collocation of testers from multiple ranges 

Organizational barriers exist between services 

Multiple ranges cooperating on a single test 

Rivalries between sites and services create barriers 

Weekly status reports are done and recorded 

Sharing information with modeling and simulation 
may lead to better models and less open air testing 

Analysis metrics and test design information are 
often shared between organizations 

Build trust with JIEDDO modeling and simulation 

Eace-to-face groups enhance sharing 

Internal review by JIEDDO 

Getting together to solve problems 

JTB employees should visit all of the organizations 
to understand how they work 

Working groups are a good start to communicating 
between ranges 

Test ranges have different priorities than the JTB 
because they have to work with other customers to 
obtain additional financial resources 

Weekly SVTCs 


Increase use of the telephone for discussions 


Table 21 presents the seventh and eighth columns of Tier III. The table 
presents considerations relevant to standards and points of contact in the design of an 
information system for the JTB. 


Table 21. Tier III Diagram: Standards and Points of Contact 


Standards 

Points of Contact 

No commonality of a strategic plan between ranges 
or from higher headquarters 

People don’t know who to ask for what information 

The lack of a written Information Sharing policy 
results in confusion through reliance on a set of 
unwritten norms that individuals interpret 
differently 

Can’t figure out who to talk to for certain 
information 

Create an information sharing policy for the JTB 

Publish a complete organizational chart that 
includes individuals, their expertise, and contact 
information 

Create process maps for data flow processes, 
infrastructure, and research & development 

Create and disseminate a point of contact list for 
each organization 

Eormalize the information sharing processes 

JTB needs to establish points of contact at each test 
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location that can help to get people the right test 
data that they need to be sharing 

Establish the processes for coordination 


There is no process in place to share information 


Create procedures for information to flow from all 
locations to a central repository 


Creating standard operating procedures within 
departments 


There are no formal procedures for information 
sharing 


No common data schema or data collection formats 


Develop overarching protocols 


Protocols are open to interpretation 


Some commonality of requirements between test 
ranges 


There is no standard method or format for 
requesting, collecting, storing, or disseminating data 


Standardize the file format of the most commonly 
used test data 



Table 22 presents the ninth and tenth columns of Tier III. The table 
presents considerations relevant to personnel turnover and the quality of shared 
information in the design of an information system for the JTB. 


Table 22. Tier III Diagram: Personnel Turnover and Quality of Shared Information 


Personnel Tnrnover 

Quality of Shared Information 

High rate of personnel turnover 

Information received is not normally timely or 
accurate, which leads to getting the information 
through different means 

Significant turnover in JTB leadership 

When sharing occurs, the amount of information 
received is inadequate 

Create a written set of guidance for continuity for 
each position in order to lessen the impact of 
personnel turnover 

Information required for budgeting is received 
quickly 


Table 23 presents the eleventh column of Tier III. The column was split 
into two for readability. The table presents considerations relevant to information sharing 
tools in the design of an information system for the JTB. 
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Table 23. Tier III Diagram: Information Sharing Tools 


Information Sharing Tools 

The JTB is creating a common server environment 
that will attach people at all locations 

Create a common data store between the JTB and 
JIEDDO 

Tying together IT infrastructure 

Create a multi-security-level database that links data 
within reports at different levels 

JTB portal home page designed as a wheel for easy 
access to all aspects of the organization 

Create a central data repository where everyone 
collects, shares, and presents data in a similar 
fashion 

Access controls on who can post to the JTB portal 
increase trust in the posted information 

A central repository should include: scenario data, 
environmental conditions, performance data, 
representation of the sensor for instrumentation, test 
plans, and test reports that constitute every data 
element or data record 

High quality hyperlinks from the JTB portal 

The enterprise lacks a good central repository that 
has all the metadata and the data required to make 
use of the information 

Read Me files are available for everything on the 

JTB portal 

Establishing a threat database to track threat 
resources and their status 

JTB portal 

An easy to use well implemented web interface 

The JTB portal should include: climate data, 
atmospheric conditions, time, space, and position 
information 

Weekly meetings 

Add a selectable pop-up notice within the JTB 
portal 

Theater Support Web Tool 

Theater Support Web Tool is not user friendly 

Creation of a database 

Transitioning to a new Oracle database 

Secure Video Teleconference 

There are issues with permissions in the information 
system 

Email (NIPR and SIPR) 

Redundant posting on the Theater Support Web 

Tool and JTB Portal 

SIPR 

Test result availability on the Theater Support Web 
Tool 

Telephone 


Table 24 presents the twelfth and thirteenth columns of Tier III. The table 
presents considerations relevant to access to information and information visibility in the 
design of an information system for the JTB. 


Table 24. Tier III Diagram: Access to Information and Information Visibility 


Access to Information 

Information Visibility 

Most people get the amount of information they 
need 

Individuals don’t know who has what information 

People get the information they need only after 
working through barriers 

Make sure everyone is aware of all available 
resources 

There is a lack of access to information from other 

Lack of visibility on updated documents 
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facilities and ranges 


There is a lack of access to documents that are 
known to be available to others 

Falsely assuming that others have access to the 
same information 

Not being able to access other ranges database for 
information. 

Working groups should do a better job of letting 
others know what information they have to share 

Obtaining raw unfiltered data from a test range is a 
challenge 

Working group members don’t disseminate what 
they learn with co-workers 

Restrict access to information to individuals who 
know what to do with the information so everyone is 
willing to share 

Inability to locate documents that are known to 
exist in other organizations 

People are reserved in large video teleconferences 
because they are not willing to share with 
information with all of the participants 


Meetings that include coalition partners restrict some 
communications 


Some organizations attempt to repress certain 
information if it might be harmful to a system 


Much of the information gathered is proprietary and 
not shared 



Table 25 presents the complete Tier II diagram. The table presents the 
information at a higher level of abstraction by categorizing the Tier III column headings. 


Table 25. Tier II Diagram: Individual Issues, Leadership Considerations, 
Management, Information Sharing, and Information Availability 


Individual Issues 

Leadership 

Considerations 

Management 

Information 

Sharing 

Information 

Availahility 

Personal 

Relationships 

JTB Level Issues 

Best Practices 

Quality of Shared 
Information 

Access to 

Information 

Individual 

Perceptions 

Leadership 

Standards 

Information 

Sharing Tools 

Information 

Visibility 


Inter- 

Organizational 

Communication 

Points of Contact 





Personnel 

Turnover 




Table 26 presents the complete Tier I diagram. The table presents the 
information at the highest level of abstraction by categorizing the Tier II column 
headings. 


54 





Table 26. Tier I Diagram: Individuals, Leadership, and Information 


Individuals 

Leadership 

Information 

Individual Issues 

Leadership Considerations 

Information Sharing 


Management 

Information Availability 


C. QUANTITATIVE RESULTS 

1. Consolidated Information Sharing Survey Results 

The quantitative data from the initial and follow-up information sharing survey 
analysis was consolidated and run through the descriptive statistics application of the data 
analysis add-on. The results are presented in Table 27. The program returned 15 
statistics for each question. The most useful statistics for this analysis were the mean, 
mode, and standard deviation. Presenting all of the survey Likert responses for each 
question was not conducive to a single summary table. Therefore, the complete 
information sharing survey text and Likert responses are included in the Appendix. 
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Table 27. Consolidated Information Sharing Survey Results 


# 

Question Text 

Summary* 

N 

Mean 

Mode** 

Standard 

Deviation 

Organizational Information Sharing Section 

m 

How often do you collaborate or share information with other organizations? 

Daily 


ESS 

1 

0.78 

2 

The main reason you share information with other organizations is: 

I want to 

12 

BBS 

4 

1.24 

3 

In youre;q3erience with JTB testing, how adequate has information sharing been between JTB organizations? 

Adequate 

m 


3 

0.78 

m 

To what extent do organizations provide relevant information to other organizations in a pro-active way, without the other having to ask for it? 

Occasionally 

m 


3 

1.23 

5 

My organization’s mission performance depends on receiving accurate information from other organizations. 


m 


5 

1.83 

6 

My organization often obtains information fromother JTB organizations required to complete the mission. 


m 


5 

1.92 

7 

My organization can depend on information fromother organizations. 

Neutral 

m 

SB 

4 

1.56 

8 

Requests for information outside of my organization are answered in a timely manner. 

Often 

m 

tliM 

4 

1.44 

m 

To what extent has information you received fromother organizations facilitated performance of your tasks? 

Often 

12 

IsiH 


1.38 

EEl 

When you obtained information fromother JTB organizations the information was: Necessary 

Rarely 



3 

1.44 

10 

When you obtained information fromother JTB organizations the information was: Complete 

Rarely 

m 


3 

1.24 

10 

When you obtained information fromother JTB organizations the information was: Accurate 

Rarely 

12 


0 

1.60 

m 

When you obtained information fromother JTB organizations the information was: Clear 

Rarely 

m 

ElM 

2 

1.19 

10 

When you obtained information fromother JTB organizations the information was: Trusted 

Rarely 

m 

Elf 

0 

1.47 

10 

When you obtained information fromother JTB organizations the information was: Timely 

Rarely 

m 

SSH 

3 

1.28 

11 

In youre;q3erience with the JTB, to what extent are you able to get the information you need fromother organizations? 


12 

I find trying to share information with other organizations and the JTB unnecessarily complex. 


m 


4 

1.34 

Intra-organization Communication Structure Section 

m 

To what extent do you observe problems caused by inadequate communication between the JTB and its organizations? 

Occasionally 



3 

1.13 

2 

The JTB information flow process allows organizations to share information adequately and in a timely manner. 

Neutral 

m 

BBS 

4 

1.47 

3 

My organization’s role in the testing process is designed in such a way that we need to interact with other organizations in order to perform our mission effectively. 




5 

1.99 

m 

My organization’s policies encourage sharing information with other organizations. 

Somewhat Agree 

12 


5 

1.73 

5 

The information flow procedures between the JTB and its organizations are clearly defined. 

Neutral 

12 

3.08 

4 

1.44 

Organizational Culture Section 

1 

My organization shares ideas openly with other organizations. 

Somewhat Agree 

12 

3.58 

5 

1.83 

2 

Other organizations can only successfully complete their mission tasks if they receive information frommy organization. 

Somewhat Agree 

12 

4.08 

4 

1.38 

3 

Other organizations often need to obtain information frommy organization to complete their mission. 

Somewhat Agree 

12 

4.17 

5 

1.40 

4 

My organization responds to requests for information fromother organizations in a timely manner. 

Often 

12 

4.00 

5 

1.54 

5 

The information gathered at my organization is unique, so no other organization would find it useful. 

Somewhat Disagree 

12 

1.67 

1 

1.37 

6 

Time spent sharing and reviewing information with other organizations could be spent more efficiently. 

Neutral 

12 

3.08 

4 

1.51 

7 

How often is information fromcurrent tests kept within your organization in order to compete for future test contracts? 

Neyer 

12 

1.25 

0 

1.82 

8 

How is the intent of keeping this information for future competition communicated? 

Explicitly 

12 

1.75 

0 

1.91 


*Suimnary indicated here is the mean rounded to the nearest \^ole number Likert response. 

**0 represents the lack of a response. Likert responses that are represented by the other numbers in the column can be referenced in the Appendix. 
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D, CONCLUSION 


1, Return to the Purpose 

The CTA and affinity diagram in this chapter used the same data points organized 
in a different manner for different purposes. The CTA presented reasons why people 
within this organization made the cognitive decision whether or not to share information. 
The affinity diagram presented the same data in a format more conducive to engineering 
a computer system to support information sharing that is customized for this 
organization. 
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V. CONCLUSIONS 


A, OVERVIEW 

This chapter presents conclusions based on the CTA and affinity diagram and 
provides recommendations for improving information sharing within the JTB and future 
research. 

B. CTA CONCLUSIONS 

1. Barriers to Information Sharing 

Barriers to information sharing were recognized in the categories of motivation, 
organizational culture, organizational environment, and organizational structure. The 
type of motivation most often recognized for causing a barrier was intrinsic hedonic 
motivation. Individuals were motivated to not share their information in order to make 
them feel better. The feelings were often in regard to maintaining a reputation, attaining 
power over others, or protecting the purity of their information. 

Organizational culture barriers were individually and collectively the most 
common data points out of all categories in the CTA. While the number of data points is 
not necessarily as relevant as their substance, it is significant that this category received 
so much attention by the participants. In the creation of the CTA tables, beliefs regarding 
sharing information related to policy or higher headquarters issues and norms were 
process and procedural issues. Individuals did not share information because the 
organization did not have the formalized policy, process, or procedures in place to create 
a culture that encouraged sharing. 

Participants recognized factors related to personal interaction as the most 
significant cause of barriers regarding the organizational environment. Few data points 
related to information systems, and none related to incentives. Personal interaction issues 
centered on the participants not sharing information or not being able to access 

information due to the lack of personal points of contact in other parts of the 
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organization. It is interesting to note that there is the same number of data points 
recognizing personal interaction as an enabler to information sharing, but for different 
reasons, which will be discussed in that section. Noting relatively few data points that 
categorize information systems as a barrier to sharing information is also significant 
when considering what aspects of the organization to fix or enhance to improve 
information sharing. The lack of data points related to incentives will be addressed in the 
future research section. 

The nonhierarchical aspect of organizational structure contained more data points 
than the hierarchical aspect. Issues centered on the dynamic and dispersed structure and 
the JTB’s lack of authority to control its suborganizations, which results in each 
suborganization being managed as a separate “stovepipe.” 

2. Enablers for Information Sharing 

Enablers for information sharing were recognized in the categories of motivation, 
organizational culture, and organizational environment. No enablers were recognized 
relative to organizational structure. Intrinsic and intrinsic hedonic motivations were 
essentially equal influences for enabling information sharing. Participants made the 
decision to share information because they felt it was the right thing to do and/or because 
it made them feel good that they were helping the war fighters. 

Both norms and beliefs were recognized as organizational culture enablers. 
Norms related to cooperation between ranges as well as between ranges and 
organizations that conduct modeling and simulation. Beliefs recognized an increasing 
leadership focus on information sharing. 

Organizational environment enablers were related to information systems and 

personal interaction. As with the barrier section, no data points were noted under 

incentives. Information systems data points mostly related to the JTB portal and provide 

a good source of feedback for developers and considerations for future improvements. 

Participants also recognized the importance of e-mail and the telephone, which ties in the 

personal interaction aspect. While personal interaction was recognized as a barrier due to 

personal points of contact, once those points of contact are made they are recognized as 
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enablers. Another aspect of personal interaction recognized as an enabler was the 
opportunity to communicate face to face, whether to create points of contact or exchange 
information between existing relationships. 

3. Participant Recommendations for Improvement 

Participant recommendations for improving information sharing were recognized 
in the categories of organizational culture and organizational environment. No 
recommendations were noted regarding motivation or organizational structure. Both 
beliefs and norms were recognized by organizational culture recommendations. 
Recommendations for beliefs centered on improving leadership and increasing 
standardization. Norms mostly dealt with different aspects of dissemination, to include 
improving processes, procedures, and visibility. 

Organizational environment recommendations were shared between personal 
interaction and information systems; again, incentives were not noted. Personal 
interaction recommendations focused on meeting in person and dissemination of a 
complete organizational chart with amplifying information. 

Information system recommendations included some specific types of information 
to be included in a central repository of information, as well as improvements for the 
current JTB portal. 

C. AFFINITY DIAGRAM CONCLUSIONS 

1. Levels of Abstraction 

The information contained in the affinity diagram provides a tool for planners and 
designers to use if JTB leadership decides that designing an information system is 
appropriate and feasible to improve information sharing. It focuses on feedback to 
planners and designers rather than understanding why members of the organization 
decide whether or not to share. As such, it is a good companion method to the CTA, but 
leads to less telling conclusions. As a tool for planners and designers, the three tiers of 
the affinity diagram essentially represent three levels of abstraction of the data that can be 
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used to customize the design of an information system to improve information sharing 
within the JTB. Tier I, the highest level of abstraction, identifies the issues that need to 
be addressed. Tier II, the middle level, guides planners in deciding what areas to focus 
on when developing requirements, an aspect of great importance prior to development. 
Tier III gets into the weeds and provides details and recommendations useful to planners 
and designers. It must be recognized, however, that not all data points included in Tier 
III are feasible or within the control of the organization. 

2. Affinity Diagram Conclusions 

Tier II is the most appropriate tier to examine when considering conclusions. It 
identifies individual issues, leadership considerations, management, information sharing, 
and information availability as focus areas. When considering how designing an 
information system can improve information sharing, leaders and planners must home in 
on what aspects technology can feasibly address. For example, it would not be feasible 
to focus the time, effort, and resources necessary to directly target individual perception. 
However, by targeting other focus areas, individual perception may be influenced. 
Narrowing the focus to targeting relevant data points under management, information 
sharing, and information availability would be a good starting point. Planners should 
also consider feasible data points from other categories and how changes in the targeted 
data points may also indirectly address other categories and data points. Once a plan is 
being created, pulling data points from Tier III will provide amplification and clarity. 

D, INFORMATION SHARING SURVEY CONCLUSIONS 

1. Organizational Information Sharing 

The organizational information sharing section results presented the level of 
information sharing efficiency between organizations. Although the average participant 
collaborated daily and the received information often facilitated task performance, the 
information was rarely complete, accurate, clear, or trusted. 
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2. 


Intra-Organization Communication Structure 


The intra-organizational communication structure section results presented each 
participants’ evaluation of how well information flows within their organization. The 
majority of participants perceived the need to interact and responded that their 
organization’s policies encouraged information sharing. 

3. Organizational Culture 

The organizational culture section results presented each participant’s evaluation 
of the organizational culture. The majority perceived the necessity of giving and 
receiving information and responded that theirs shares openly. 

E, RECOMMENDATIONS 

1. Overall 

CTA provides focus areas for leadership to address when considering how to 
increase the likelihood of members sharing information. The information sharing survey 
enables the participants to evaluate multiple aspects of information sharing between and 
within organizations. The affinity diagram is a tool that provides a starting point for 
designing an information system that enables the members to share information by 
enhancing what they have identified as the strengths and improving what they have 
identified as the weaknesses. 

2. CTA Recommendations 

This section presents the author’s recommendations to address the most common 
barriers to and enablers of information sharing within the JTB. User recommendations 
presented earlier in this chapter should be considered when implementing and changes 
relative to information sharing. 

The most common barriers were organizational culture, both beliefs and norms, 
and personal interaction. In order to mitigate those barriers, the researcher recommends 
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that the JTB create a position to focus specifically on information flow both within and 
outside of the organization. The position would initially be concerned with improving 
information sharing by recognizing and mitigating barriers by being a ‘go to’ person for 
information dissemination and access issues. 

Areas recognized as enablers that should be enhanced include the organizational 
culture norm of cooperation between test ranges and the information systems aspect of 
the organizational environment. To enhance cooperation between ranges, this researcher 
recommends increasing meetings between testers from different ranges. Including the 
actual testers is significant because they are the people who generate the information that 
everyone else will share. If they create a culture that includes sharing, it will influence 
the motivation of others and the norms of the organization. Also, the positive image of 
information systems by the participants leads to the recommendation to enhance the 
capability of information systems to share information. Investing in JTB portal 
improvement and including some of the user recommendations would increase sharing 
capability and let the members of the organization know that their leadership values their 
input. 


3, Affinity Diagram 

Information sharing within the JTB would improve if JTB leadership provided a 
system that could enhance building personal relationships and standardize the 
information sharing process in order to improve its quality and timeliness, while 
improving information visibility and access. Whether or not that is feasible is up to the 
JTB leadership. A recommendation more applicable at this point in the evolving status of 
the JTB is that any improvements to information systems within the JTB should at least 
include a review of the affinity diagram because it is customized for the JTB. Other 
organizations can reference the diagram as an example of how to customize their 
information technology strengths and weaknesses. 
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4. 


Information Sharing Survey 


Results from the information sharing survey reinforce the recommendation to 
create a position to focus specifically on information flow both within and outside of the 
organization. Individuals collaborate daily, perceive the need, and perceive information 
sharing positively. Having an individual focus on making the information flow more 
efficient would improve both organizational culture and performance. 

5. Future Research 

Future research on incentivizing information sharing for JTB members may prove 
useful. None of the participants provided data points that related to incentives and there 
were few data points related to extrinsic motivation, but that was likely due to the focus 
of this research being to identify barriers and mitigate them. Because there is no 
incentive system, it is not a barrier, but follow-on research could focus on incentive 
systems for government employees and contractors. While intrinsic and intrinsic hedonic 
motivations are powerful, there is always a point where some extrinsic motivation can 
make the difference between deciding whether or not to share. 
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APPENDIX. INFORMATION SHARING SURVEY TEXT 


The text of the information sharing survey distributed to all participants is presented helow. It 
provides clarification on the statistical mean column of the Consolidated Information Sharing 
Survey Results table presented in Chapter IV. 

Organizational Information Sharing 


1. How often do you collaborate or share information with other organizations? 


0 

1 

2 

3 

4 

No Response 

Daily 

Weekly 

Monthly 

Quarterly 


2. The main reason you share information with other organizations is: 


0 

1 

2 

3 

4 

No Response 

I don’t 

I was directed to 

Everyone else 
does 

I want to 


3. In your experience with JTB testing, how adequate has information sharing been 
between JTB organizations? 


0 

1 

2 

3 

4 

5 

No 

Response 

No 

information 
has been 
shared 

Too much 
information 
sharing 

Inadequate 

Adequate 

Excellent 


4. To what extent do organizations provide relevant information to other organizations in 
a pro-active way, without the other having to ask for it? 


0 

1 

2 

3 

4 

5 

No 

Response 

Never 

Rarely 

Occasionally 

Often 

Always 


5. My organization’s mission performance depends on receiving accurate information 
from other organizations. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

disagree 

Somewhat 

disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 
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6. My organization often obtains information from other JTB organizations required to 
complete the mission. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

disagree 

Somewhat 

disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 


7. My organization can depend on information from other organizations. 


0 

1 

2 

3 

4 

5 

No Response 

Strongly 

disagree 

Somewhat 

disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 


8. Requests for information outside of my organization are answered in a timely manner. 


0 

1 

2 

3 

4 

5 

No 

Response 

Never 

Rarely 

Occasionally 

Often 

Always 


9. To what extent has information you received from other organizations facilitated 
performance of your tasks? 


0 

1 

2 

3 

4 

5 

No 

Response 

Never 

Rarely 

Occasionally 

Often 

Always 


10. When you obtained information from other JTB organizations the information was: 
Necessary: 


0 

1 

2 

3 

4 

No Response 

Never 

Rarely 

Always 

N/A 


Complete: 


0 

1 

2 

3 

4 

No Response 

Never 

Rarely 

Always 

N/A 


Accurate: 


0 

1 

2 

3 

4 

No Response 

Never 

Rarely 

Always 

N/A 
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Clear: 


0 

1 

2 

3 

4 

No Response 

Never 

Rarely 

Always 

N/A 


Trusted: 


0 

1 

2 

3 

4 

No Response 

Never 

Rarely 

Always 

N/A 


Timely: 


0 

1 

2 

3 

4 

No Response 

Never 

Rarely 

Always 

N/A 


11. In your experience with the JTB, to what extent are you able to get the information 
you need from other organizations? Results were integrated with other qualitative data 
from the initial round of data collection. 

12. I find trying to share information with other organizations and the JTB unnecessarily 
complex. 


0 

1 

2 

3 

4 

5 

No Response 

Never 

Rarely 

Occasionally 

Often 

Always 


Intra-organization Communications Structure 


1. To what extent do you observe problems caused by inadequate communication 
between the JTB and its organizations? 


0 

1 

2 

3 

4 

5 

No Response 

Never 

Rarely 

Occasionally 

Often 

Always 


2. The JTB information flow process allows organizations to share information 
adequately and in a timely manner. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 
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3. My organization’s role in the testing proeess is designed in such a way that we need to 
interact with other organizations in order to perform our mission effectively. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 


4. My organization’s policies encourage sharing information with other organizations. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 


5. The information flow procedures between the JTB and its organizations are clearly 
defined. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 


Organizational Culture 

1. My organization shares ideas openly with other organizations. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 


2. Other organizations can only successfully complete their mission tasks if they receive 
information from my organization. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 

3. Other organizations often need to obtain information 
complete their mission. 

from my organization to 

0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 
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4. My organization responds to requests for information from other organizations in a 
timely manner. 


0 

1 

2 

3 

4 

5 

No 

Response 

Never 

Rarely 

Occasionally 

Often 

Always 


5. The information gathered at my organization is unique, so no other organization would 
find it useful. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 


6. Time spent sharing and reviewing information with other organizations could be spent 
more efficiently. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 


7. How often is information from current tests kept within your organization in order to 
compete for future test contracts? 


0 

1 

2 

3 

4 

5 

No Response 

Never 

Rarely 

Occasionally 

Often 

Always 


8. How is the intent of keeping this information for future competition communicated? 


0 

1 

2 

3 

4 

No 

Response 

Implicitly 

Explicitly 

Both 

No information is kept for 
future competition 
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